Hexagonal Rails Controllers

I’ve had a long love-hate relationship with Rails. I love the MVC framework and how it’s improved our speed of writing web apps. But I’ve never really been completely happy with it. I don’t generally agree with most of its opinions. I prefer models that follow the Data Mapper pattern, not the Active Record pattern. This includes separating the persistence layer from the models’ business logic. I prefer Slim or HAML to ERB. I prefer RSpec to Test::Unit or MiniTest. When Merb hit the scene, I was ready to make the jump, until Merb merged with Rails.

So inspired by PJ Hagerty’s recent article on alternative Ruby web frameworks, I started thinking about how I’d write a replacement for Rails. I’d definitely keep the basic MVC framework. But I’d also want to implement a more hexagonal architecture.

I started sketching out what this would look like, but I ended up starting with a Rails controller and finding the simplest way to make it hexagonal. I really don’t like callbacks, because they make tracing program execution difficult. But I didn’t see any other alternative. I found a simple pub/sub Ruby library called Wisper. It literally has only publish, subscribe, and on methods. (You use on to register single callbacks via blocks, and subscribe to register an object with method names corresponding to the callback names.)

The trick was figuring out how to break the controller into 2 pieces. What finally helped me was to find the single responsibilities of the 2 pieces. The Rails controller would remain in charge of managing the web interface, but would delegate to the other piece to handle any application-specific business logic. I decided to re-watch Uncle Bob Martin’s “Architecture The Lost Years” talk, which was the first time I was introduced to the ideas of Hexagonal Architecture. (He doesn’t name the architecture in the talk, but later calls it Clean Architecture.) He does a decent job of explaining how to break these 2 pieces apart. He used the term “interactor” in that talk, so I decided to go with that. He said that Jacobsen calls it a Control Object in Object Oriented Software Engineering, but that’s too close to Rails’s “controller”.

So here’s an example of what I ended up with:

class OrderController < ApplicationController
  def index
    interactor.on(:display) { |orders| render orders }
    interactor.list
  end

  def show
    interactor.on(:display) { |order| render order }
    interactor.on(:not_found) { |order_id| render status: 404 }
    interactor.get(params[:id])
  end

private

  def interactor
    @interactor ||= OrderInteractor.new
  end
end
require "wisper"
require "order"

class OrderInteractor
  include Wisper.publisher

  def list
    orders = Order.all
    publish(:display, orders)
  end

  def get(id)
    order = Order.find(id)
    publish(:display, order)
  rescue ActiveRecord::RecordNotFound
    publish(:not_found, id)
  end
end

I do have a few problems with this solution though. I’m not a fan of the name “interactor” for the business logic. I thought about calling it OrderOperator, or maybe OrderOperations, because it’s really a collection of operations. Perhaps it would be better to separate each operation into a separate class. Trailblazer does it that way. And for more complicated business logic, I would do that too, using the Method Object pattern. But like a Rails controller, there’s a lot in common among all the operations. I feel like a separate class for each operation for each would create too many coupled classes.

I’m also uncomfortable with the fact that the controller is delegating almost everything to the interactor. I guess this is OK, but it feels like there’s too little left when every line starts with interactor. I suppose extracting things some more would help mitigate this concern I’ll likely write a small gem to perform that extraction. I expect that that will allow a typical controller to be written in only a few lines. And maybe the same for the interactor side.

With the business logic extracted out of the controller, it was really easy for me to write a command-line version of the app. As Uncle Bob says, “the web is not particularly important to your application.”

I’ve put the code for this example on GitHub: https://github.com/boochtek/hexagonal-rails. I’ll likely experiment with it some more over the next few weeks and months.

Resolutions

January kept me pretty busy, so I’m a little late to this. But better late than never. And as an Agile practitioner, I don’t think personal retrospectives should be limited to one time of year.

Review of 2014

Last year I wrote a blog entry listing my goals for 2014. As far as New Year’s resolutions go, I was relatively successful — about 50% of my goals accomplished. Unfortunately, my Open Source contributions weren’t as strong as I had hoped; while I released some of my own work, I didn’t do much else. I did increase my blogging; getting in on a weekly blogging pact helped immensely. I also increased my participation on the This Agile Life podcast to a level that I’m happy with. But the accomplishment I’m most proud of was giving a presentation at RubyConf.

Plans for 2015

I’d like to keep things rolling from last year, but crank up a few things. My plans are quite ambitious, so I don’t expect to get everything done by any means. But I think by setting the bar high, I’ll end up with a lot I can be proud of.

Job Hunting

Late last year, I took the jump into independent consulting. So far, I’ve really enjoyed it, and I’m booked up through April. My wife graduates in May, so we’ve got the possibility of moving if that makes sense. So I’ll be looking for consulting projects in town, but I’ll also be looking at jobs in San Francisco and Chicago. The possibilities are exciting, and I’ll be taking my time to find something just right.

Conferences

I was incredibly nervous leading up to my RubyConf presentation. Part of that was just the common fear of public speaking. For me, that only kicks in at around 100 people, and this audience was around 250. I think another reason was that I chose a really ambitious topic, and I kept finding more that I wanted to talk about, but wasn’t prepared for. But I think I did a pretty good job presenting an advanced topic. And I was so pumped by the sense of accomplishment as soon as I finished. So I’m hoping to do it more. I’ve already submitted a couple proposals, and plan to submit several more.

Blogging

I believe that blogging is important for me to get my thoughts down — for myself and to share with others. I was really successful last year when I had a partner to keep me honest, via a pact. So I’ve started up another pact this year, which will hopefully ensure I’ll keep things going. I’ve got a really long backlog of topics, so as long as I keep at it, I’ll have plenty to write about.

I also want to move away from WordPress to a static system — probably Middleman. I’ve got 2 major problems with WordPress. First, I no longer trust its security, nor the security of any application written in PHP. Second, it generates HTML every time someone requests a page, instead of when the content is updated. I find that to be a waste of resources, and problematic from a security standpoint. The main problem with moving to a static blogging system is that I really want to allow comments, pingbacks, and tweetbacks. So I’ll have to find a way to integrate those.

Programming Language Design

Last year I started thinking about programming language design, and started implementing a language tentatively called Brilliant. I’ve done a lot more thinking on the topic, and have a lot of notes. But I haven’t implemented much more yet. This year, I’d like to get my thoughts more organized, and write a series of blog posts on various aspects of language design. The most interesting part seems to be the trade-offs involved in the ways that various language features interact. So I’d like to make some progress on the language implementation, but more importantly, I’d like to get a lot of my design ideas written down.

I’m also going to spend a lot of time learning a bunch more programming languages, so I have a better understanding of possible features, combinations of features, and their interactions. I’ve already start with Elixir, Clojure, and Racket. I’m hoping to also look at OCaml, Factor, and Haskell. I’ll probably also take a look at the 2 “Seven Languages in Seven Weeks” books.

Agile Book

I think people often have trouble getting started with Agile. I started on a book last year, and got down quite a lot of good ideas. But I realized that I’m going to have a hard time organizing all those ideas into something coherent. Still, I’d like to try to get something out there that lets people get started with Agile. My idea is to present a toolbox of practices to get started with and build on that foundation over time with additional practices. Sort of a playbook on how to get started over the first 6 to 12 months and be successful. I want to make some progress on the book, at least enough to decide whether it’s worth the effort to finish it and self-publish it.

 

TDD Is Alive And Well

I went to RailsConf this year, and the very first talk was a keynote by David Heinemeier Hansson (DHH), the creator of Ruby on Rails. The TL;DR of his talk was “TDD rarely has value”. He followed up with a blog post the next day, titled “TDD is dead. Long live testing.“, and 2 more posts. I think this line of thought is terribly misguided, and causing more harm than good. This article is my response.

First, I would like to address the good points of the talk. He said that programming is pseudoscience, and that people want to tell us that there’s a secret to being a better programmer. But what it really takes is working hard — reading a lot of code, writing a lot of code, and rewriting a lot of code. He’s right. And I also agree with him that you should forget about patterns for a while when learning to code. Beginners try to throw patterns at a problem instead of letting the patterns emerge where they’re supposed to.

I don’t completely agree that programming is a pseudoscience. In some ways it is, but I think it’s more of a craft. It’s a craft, because there’s a lot of science involved, but there’s also an art to doing it well. And like any craft, you’re always working to get better. So to respond to DHH’s stance that “software is more like poetry than physics”, I think it falls
somewhere in between.

With regard to the software engineering practices we use, there really isn’t much science available, mostly because it’s a soft science. That is, it’s really hard to isolate a single variable when comparing code between projects. And nobody has the time or money to write the same code so many times that the differences would be statistically significant.

So we don’t have much science on TDD. But we do have some. Here’s a collection of several: StudiesOfTestDrivenDevelopment. And here’s one that explicitly looks are the difference between test-first and test-last: Does Test-Driven Development Really Improve Software Design Quality? What do these tell us? They tell us that TDD costs us about 10-30% in short-term productivity; reduces bugs by 30-90%, and decreases code complexity by about 30%. As Code Complete tells us (in section 20.5, with studies to back it up), improving quality reduces development costs. So, like most Agile practices, this is a case where spending a bit more time in the short term leads to time savings in the long term.

The more important lesson in the talk was that you have to do what works best for you and your situation. If TDD doesn’t give better results, then either find out how to make it give better results, or stop using it. As we often say in the Agile world, Agile doesn’t mean that you can stop using your brain. While I think TDD is appropriate in most situations, there are cases where it’s not worth the additional up-front cost. If the most important thing for your project is time-to-market, then not testing might be the right decision for you.

To me, TDD provides a bunch of benefits. First and foremost, TDD is a design discipline. It ensures that I think about how my code will be used before I think about how to implement it. This is very powerful in ensuring that the code is well-written from the perspective of other code using it.

Tested code provides confidence to be able to make changes without breaking things. If we write tests after the code, we’re less likely to write them. Tests written after the code also tend to test the implementation instead of the desired functionality. What we really want is tests written as a specification. With tests as a specification, we can come back later and understand why code was written. Without tests, or with poor tests, we can’t understand why the code is there; if we want to rewrite it, we don’t have the confidence that we’re not missing something. Writing tests first also ensures that we only write the code that is needed to implement the required functionality.

I’m not sure why DHH hasn’t “gotten” TDD. I’m not sure if it’s because he’s  a better coder than average, or if he just thinks in a different way than most of us. I think it’s partly because he doesn’t understand TDD, which he admitted might be the case. And I think he’s conflating TDD and unit testing.

DHH is influential in the developer community, especially those newer to Ruby and Rails. People listen to what he has to say. I was happy to see almost every other speaker made fun of DHH’s ideas, and most of the crowd knew better. But there will be a lot of others who will hear DHH, respect his opinions, and not give TDD the try that it deserves. And that’s sad, because it will lead to an overall reduction in code quality in the world.

Here are some other people’s thoughts on the matter:

 

Ruby Pattern: Parameterized Module Inclusion

I’ve run across a pattern in Ruby lately that I really like. It solves some problems that I’ve struggled with for several years. Let me start with the problems.

Let’s say you want to include an ORM in a model class, and want to tell it what database table to use. Typically, you’d do this:

class User
  include MyORM::Model
  table 'people'
end

But that table call is more like an option to the module inclusion than anything else. So what we’d really like is something like this:

class User
  include MyORM::Model, table: 'people'
end

But that’s not valid Ruby; include doesn’t let you pass anything other than a module.

So when I was learning about Virtus, I noticed that its example of how to include it is a bit different than the standard Ruby idiomatic include:

class User
  include Virtus.model
end

At first glance, it reads like the first example. But on closer inspection and consideration, it’s quite a bit different. Where MyORM::Model is a constant that refers to a module, Virtus.model is a method call. So there’s a method named model in the Virtus module. That method returns another module — which is exactly what’s needed in order to include it into our model class.

The easiest way to implement Virtus.model would be this:

module Virtus
  def model
    ::Virtus::Model
  end
end

module Virtus::Model
  # ...
end

If Virtus.model doesn’t need to take any arguments, that’s perfectly fine. In fact, I’ve started to use this implementation of the pattern for modules that don’t need parameters.

Because Virtus.model is a method, we can also call it with options:

class User
  include Virtus.model(constructor: false, mass_assignment: false)
end

We could even pass a block. But how do we process those options? There are a few different ways. However we do it, we have to be sure to return a module. And we can create modules in a few different ways.

Virtus uses the builder pattern. It takes the parameters passed in and builds a module dynamically. By that, I mean that it calls Module.new and then adds methods to that module. It does this by mixing in other modules, but it could do it by dynamically defining methods as well.

I’ve never seen this pattern in any other language. It’s obviously only possible because we can dynamically create modules.

The use of this idiom seems to be catching on a bit in the Ruby community. I’ve started using it myself, and will be adding it to my Includable::ActiveRecord gem soon.

 

 

Brilliant – My Very Own Programming Language

I’ve decided to design and implement my own programming language, and call it Brilliant.

I’ve been interested in programming languages and linguistics almost as long as I’ve been using computers. I’ve long thought that if I ever go back to college, it’s likely that I’ll concentrate on programming languages as a specialty.

My recent discovery and involvement with the Crystal programming language has gotten me excited about new language ideas. It’s also helped me realize that implementing a language myself is feasible, and that there’s no better time to start than now.

Implementation

For now, the Brilliant implementation doesn’t let you do much more than “Hello World”. But you gotta start somewhere. So this article isn’t really “Introducing Brilliant” so much as the start of a series of articles on its design and implementation.

I wanted to use a PEG (parsing expression grammar) parser for several reasons. For one, they seem to have gained a lot of popularity in the past few years. Also, PEGs cannot be ambiguous, which can solve a few difficult problems, such as the “dangling else“. Perhaps my favorite feature of PEG grammars is that you don’t need a separate tokenizer (lexer). This provides a nice advantage in that we can use keywords like “class” as variables, as long as they’re not used in a place where the keyword would make sense.

So knowing that I wanted to use a PEG, I had to find a PEG parser. I kind of wanted to use ANTLR, which has been a leading parser for many years. But the PEG seems to be new to version 4, and I couldn’t find any Ruby bindings for version 4. Treetop seems to be the most popular parser for Ruby, but I found the EBNF format that Rattler uses to be more to my taste. I think the fact that it’s newer also gives it a few advantages, having had a chance to learn some lessons from Treetop.

I thought about using the Rubinius VM, but decided to go with LLVM, mainly since it has slightly better docs for Ruby, and because it’s what Crystal uses. Also, it’s pretty easy to get it to compile to a binary executable or run in a JIT. In the future, I might consider switching to the Rubinius VM, the Erlang VM, or the Perl 6 VM (Parrot). But for now, I like the idea of being able to compile to a binary and easily interface with C, just like Crystal.

Goals

My main goal is to have fun playing around with language ideas.

I’ve found a really great language in Ruby, so I’ll be using it as a starting point. But Ruby does have its faults. In some ways, I want to answer the question “what would Ruby look like if we designed it today?”.

But I also want to explore other ideas. What if objects defaulted to immutable? What if functions and methods were assumed to be pure by default? Might it be possible to infer the purity of a function or method? (If so, we could automatically memoize them.) Can we make creating an actor as easy as creating an object?

I’ll also be looking at ideas from other programming languages. Could we get some of the benefits of Haskell’s purity without having to do somersaults to do IO? Could we mix Python’s indentation style scoping with brace style or begin/end style? Could we integrate Icon’s ideas about success and failure (goal-directed execution)? What interesting features can we pull from Ada, Io, CoffeeScript, Crystal, Rubinius, Perl 6, etc.?

I’m not so much as interested in cutting-edge features, as in features that can be easily used by the average programmer. More importantly, I’m interested in how features interact with each other, so they fit well together to create a whole that’s greater than the sum of the parts.

Naming

I wanted to name my programming language “Fermat”. I’ve long been intrigued by Fermat’s Last Theorem, and Fermat’s Little Theorem is important in number theory. Unfortunately, there’s already a computer algebra system with that name.

So I decided to find a name in the same vein as “Ruby” and “Crystal”. I looked at the Wikipedia page for “gemstones” for some inspiration. A lot of the names of gemstones are already taken. I considered some obscure gemstones, but saw the word “brilliant” and thought it was decent. It’s not the name of another gemstone, but still evokes some similarity to Ruby and Crystal.

So that’s the name for now. Perhaps I’ll decide to change it at some point in the future, but I needed a name for the project, as well as a file extension for source code files. I chose “bril” for that. I suppose “br” would be a better choice. Perhaps I’ll change that before the next article in this series.

Future

I hope to work on Brilliant every once in a while. I expect it’ll take a couple years before it’s really very useful.

When I do add a major feature, I’ll be certain to blog about it. I’ve got tons of ideas strewn about in various files. It would be great to get them organized and published —and even better to get them implemented.

Estimation Isn’t Agile

I don’t believe that estimation should be part of any Agile practice.

One of our managers recently mentioned that we hadn’t met the “contract” that we had “committed” to in our last iteration. This was complete nonsense, because A) we hadn’t made any such commitments, and B) we completed many more story points than the previous iterations (and without inflating story points).

estimates-as-deadlines

But her language made me come to several realizations. First and foremost, estimates are contracts. Sure, they’re not supposed to be treated as commitments, but they almost always are. And what does the Agile Manifesto say about this? It says that we should value customer collaboration over contract negotiation, and responding to change over following a plan. So it’s pretty clear that treating estimates as commitments is completely counter to the Agile values.

Why does this matter? What benefits do the Agile values bring us? I think the biggest benefit they bring is changing the way that we work, so that we can better deliver value to our customers. Without Agile, we’d just keep working the way we’ve always done things. And that didn’t seem to be working out so well. If we follow the Agile values and principles, at least we’ll have a fighting chance of improving our ability to deliver value.

Ask yourself — have you ever seen a software development project that was on time and on budget? Where the estimates were spot-on? Of course not. For one, we’re terrible at estimating. For another, our plans change — either from external factors, or from what we learn as we go.

Improved Estimation

To me, Agile is also about facing reality — and embracing it. It realizes that we’re terrible at estimating. It realizes that plans change. Most Agile methodologies have some tricks to counteract Hofstadter’s law. Generally, we use relative story points instead of hours, and then use an empirical factor to convert points to hours.

When this works, it is better than any other estimation I’ve ever seen. But it doesn’t work very often. People have trouble with relative estimation. How do you set the basis for what a point means without relating it to actual hours? Affinity estimation could work, but then you have to remember what the basis was. We’ve got a large distributed team, and when we tried this, we couldn’t all remember what the basis was.

Since we couldn’t get affinity estimation to work, we tried changing to perfect hours (only powers of 2). But then people thought of them as time. When we took longer than the estimate on an individual story, managers and team members thought we were taking longer than we should have. So our estimates ended up causing problems.

What Can We Do Instead?

Managers want estimates so that they can have predictability. They want to know when new features will be available. Is there a better way to get what we need?

I believe there’s a better way — prioritization. If you work on the most important thing first, then the most important thing will get done first. We should always be working on the next most important thing.

What if there’s more than 1 thing that’s most important? Then you’ve failed. You’ve failed at logic if you can’t understand that only 1 thing can be most important. You’ve failed at prioritizing the customers’ needs. You’ve failed at project management.

Arguments

1. Why can’t you just tell us how long it will really take?

Because we don’t know. Because we can’t know. This is the first time we’ve ever implemented the functionality you’ve asked for. If we’d done it before, we’d just use that existing code. As Glenn Vanderburg pointed out in his excellent talk on Software Engineering, we’re not building software, we’re architecting it.

2. But we have to tell our customers what to expect.

Why? Is the product so bad that you can’t keep customers around without leading them on with future enhancements? And why do customers need exact dates? A general roadmap telling them what the priorities for upcoming features should be sufficient.

3. But we have to have messaging about new features.

OK. Then send out that messaging once the feature has made it to Staging. Or even after it’s been rolled out to Production.

4. But we’ve promised these new features to the customers by this date.

Ah, so you’ve made promises to the customer that you don’t have control over. Have you ever heard of “under-promise and over-deliver”? That’s how you create happy customers. Yet you’ve done just the opposite, haven’t you? And then you want to blame someone else.

Risk

Estimates are risk. But the risk doesn’t come at the end, when the estimates are shown to be incorrect. The risk was in asking for the estimates in the first place, and placing trust in them. Don’t do it. Don’t promise things that you can’t be sure of.

Embrace this reality. Embrace this uncertainty. Always focus on what’s most important. That’s how you make customers happy.

Slow Down!

There’s a tweet that I saw recently, with some simple advice for novice programmers:

Slow down.

This is probably good advice for most programmers. Our team recently noticed that every time we try to rush things, we make mistakes. And the mistakes end up costing us more time than if we had just done things at our normal pace. Slowing down ensures that you do things right, and when you do things right, you end up with a higher-quality product.

Speed and Code Quality

There are 2 types of code quality: internal and external. External code quality can be measured by how many bugs have been reported by customers. Internal code quality is harder to measure, but it mainly deals with the ability to change the code. When your internal quality is low, you’ve got lots of technical debt, and it’s harder to make changes.

So when you try to write code quickly, code quality decreases, leading to a code base that takes more time to make changes to. Conversely, when you slow down, your code quality improves, and it becomes easier to make changes more quickly. So when writing code, slowing down in the short run leads to a speed-up in the long run.

Speed and Process Improvement

But writing code isn’t the only place where we try to speed up. On an Agile team, we’re always trying to improve the way we work — especially at the beginning stages of an Agile transformation. So we’re eager to make changes in our processes. But I’d urge you to slow down here as well.

My colleague Amos and I frequently argue over pair switching. It’s funny, because we agree on everything except for 1 small detail. We both think pair switching is very important, to ensure that team members see more of what’s going on, to bring more ideas to each story, to prevent knowledge silos, and to encourage team ownership. Where we disagree is how long an ideal pairing session should last. I think pairs should switch every 2 hours, and he thinks 1 hour is ideal. I’ve seen teams reach the 1 hour pairing sessions successfully. But usually not without some pain and even often failing at the first attempt.

There’s nothing inherently wrong with failing. But if you fail at something, you’re not likely to try again. After all, you should learn from your failures, right?

So if you want your team to do something, you probably don’t want them to fail at it. If they fail, they won’t want to try a second time. That’s just human nature, and learning from failure. While you might think that they failed because they weren’t ready for the change yet, they’ll most likely think that they failed because this particular change won’t work for their situation. And they probably won’t know what to change when trying again, so they won’t try again.

I’ve seen this over and over. Back when Linux was up-and-coming, when a consultant pushed a company into using Linux before they were ready for it, and it didn’t work out, that company was cautious about trying again. So instead of being on the leading edge of using Linux, or even the middle of the pack, they ended up more toward the trailing edge. Had they not been pushed, they would have gotten more benefit in the long run.

So my advice in process improvement is the same as in programming: slow down. Take small steps toward what you think is the ideal. Make a small change, see how it works out, and adjust. As long as you’re still moving in the right direction, I believe you’ll move faster by taking small steps than by trying to make big leaps.

Burying the Lede

Most of us don’t write very readable shell scripts. There are plenty of things we could do better, but today I want to talk about one in particular — burying the lede.

The term “burying the lede” comes from the field of journalism. Here’s the Wiktionary definition:

To begin a story with details of secondary importance to the reader while postponing more essential points or facts.

Like a good news article, code should tell a story. And the story should start with what’s most important. In the case of code, the most important information is the high-level functionality — a succinct summary of what the program does. In other words, write (and organize) the code top-down, as opposed to bottom-up.

Unfortunately, shell script doesn’t make this easy. Due to the way shell scripts are interpreted, you can’t call a function until after you’ve defined it. This leads to most of us structuring our code like this:

function do_something { ... }
function do_something_else { ... }

do_something
do_something_else

The problem with this is that the function definitions will likely take quite a few lines, and we won’t see what the top-level functionality is until we reach the end of the script.

I’d like to propose a standard way to structure shell scripts to mitigate this issue. (I’m really only talking about shell scripts that have function definitions within them.) I’m sure I’ve seen a few scripts do this, but it’s not very common at all.

My proposal is simple:

function main {
  do_something
  do_something_else
}
function do_something { ... }
function do_something_else { ... }

main

This structure lets us start with the lede. We describe the top-level functionality right away. Only then do we get to the secondary details. The name main makes it pretty clear that it contains the top-level functionality.

I’ve recently started writing my shell code like this, and I’m happy with the results. I’ve also started to use some other programming techniques in my shell scripts to improve readability: better naming, extracting more methods, and moving helper methods into separate files. It feels good to treat shell scripts like real code instead of just some stuff I’ve hacked together.

PS. The WordPress theme I’m currently using (Twenty Eleven) also buries the lede — I can barely even see the title of the blog post on my screen without scrolling. I’m going to have to change that soon.

Yak Shaving #1: Cursor Keys

I recently decided to start using Emacs again. I used it extensively from the early 1990s until the early 2000s. I pretty much stopped using it when I had a sysadmin job with no Emacs on the servers, and no ability to install it. With the rising popularity of tmux and tmate for remote pairing, and my dislike for vim’s modes, I decided to try going back to Emacs in the terminal.

One thing I really want in a text editor is good cursor key support. Shifted cursor keys
should select text, and Control+left and right should move by words. (Apple HIG says to use Option+left and right to move by words; most apps on Mac OS X seem to support both.) Things have worked this way on almost every text editor on every OS I’ve used — Amiga, DOS, Windows, NeXT, Mac, Motif, Gnome, KDE. It’s a part of the CUA standard that’s been in common usage on everything since the mid-1980s.

Enabling cursor keys in Emacs was pretty easy. I’ve decided to use Prelude to make getting started with Emacs easy. Emacs comes with the cursor keys enabled, but Prelude disables them. Undoing Prelude’s change is pretty easy:

(setq prelude-guru nil)

Trying to make shifted cursor keys work is where the trouble began. They work in the GUI version of Emacs, but not from the terminal. It turns out that the Mac Terminal doesn’t distinguish between cursor keys and shifted cursor keys in its default configuration. So I had to figure out how to configure key bindings in Terminal.

That’s easy enough — they’re in the preferences. But what should I set them to? This took a lot of research. Terminal emulation and ANSI code sequences are obscure, complex, and inconsistent. I eventually found the info I needed. For starters, Shift+Right, Shift+Left, Shift+Home, and Shift+End are defined in the terminfo. The rest I was able to piece together from various sources around the Internet.

I’m also trying to script my Mac configuration. So instead of manually adding all the keybindings in the Terminal preferences pane, I decided to write a script. Mac OS X does a decent job of allowing you to change preferences from the command line. For example, to always show the tab bar:

defaults write -app Terminal ShowTabBar 1

Easy enough, except for a couple problems. First, I had to figure out the obscure syntax used in the preferences for key codes. I was able to piece these together with some more Internet research. But the really big problem is that the keyboard bindings are 2 levels deep within a “dictionary” (hash map). And the defaults command doesn’t handle that. There are some obscure utilities that handle nested preferences, but they don’t work well with the preferences caching in Mac OS X 10.9 — a problem I ran into while testing.

So now I’m writing a utility in Python that does what the defaults command does, but that will handle nested dictionaries.

There’s a term for this process of having to solve one issue before you can solve another, and the issues get several layers deep. It’s called yak shaving.

Here’s another good example:

Father from Malcolm in the middle having to fix one thing before fixing another, ad infinitum.
Yak shaving – home repairs

I’m sure this will be the first of many posts about me yak shaving.

Chording Keyers

I’m considering buying a chording keyer. A keyer is a 1-handed text input device. Chording means that you can hit more than 1 key at a time. I’ve been interested in these for a long time actually. They’ve been popular in the wearable computing (“cyborg”) field, but never caught on anywhere else. I’ve always thought that 1-handed chording keyer would be a great input device to use with a cell phone. It’d be even better with a heads-up display like the Google Glass.

There are quite a few chording keyers available. It doesn’t appear that any will meet exactly what I’m really looking for, but I might take the leap anyway. It should be a hand-held device. I really want a Bluetooth device, so I can use it with a computer, phone, or tablet. Below are the ones I’m considering.

Twiddler

The Twiddler has been around quite a while, as probably the main commercially available keyer for wearable computing. There was a time that it was unavailable, but there’s a new version available from HandyKey now. It’s $200 ($220 Canadian), and comes in only USB.

The Twiddler looks like a small TV remote. The current version (2.1) has 4 thumb keys, a thumb joystick (for mouse), and 12 small finger keys. It has good community support, including an alternate chord layout.

In10did Decatxt

In10did has been trying to bring out a few variations of their devices. The Decatxt looks like a pack of cigarettes, with 8 keys on the front and 2 thumb keys. It can be used with 1 or 2 hands. All letters can be typed with 1 finger or 1 thumb and 1 finger.

The Decatxt has USB and Bluetooth models, but I’m unable to find pricing or a way to buy one. It looks like there have been no updates for over a year, so I’m concerned that this is going to be vaporware.

CyKey

The CyKey is based on the Microwriter, which goes back to the early 1980s. The Microwriter was a desktop-based device, but it looks like the CyKey might be able to be used as a hand-held device. It has 9 buttons on the face, arranged in 3 groups of 3 buttons.

The CyKey is $125 (£75) and works over InfraRed connected to USB. A Bluetooth model is supposedly in the works.

EkaPad

The EkaPad is like a cross between the Decatxt and the Twiddler. It has 12 buttons on the face, but no thumb keys. It’s got a nice well-thought-out design that looks quite ergonomic.

The EkaPad is $150, available in USB only. A desktop holder is available to hold it upright, or it can be used hand-held.

FrogPad

The FrogPad is meant to be used on a flat surface only. It looks kind of like a third of a laptop keyboard. It’s got 15 regular keys and 5 modifier keys. Only 1 key and 1 modifier can be chorded. Right-handed and left-handed models are available, with USB connectivity.

The FrogPad2 was recently announced, replacing the original FrogPad. It switches to standard desktop-style keys, and adds some more keys along the top. It supports USB On-the-Go and Bluetooth. I’m not really interested in this model with its regular keys, but it doesn’t appear that the original version is being made any longer. The new models are priced at $200. I believe the old models were $150.

Chordite

The Chordite looks the most promising, but it’s only a concept with some DIY prototypes. It’s a hand-held device, with 2 buttons on each of the 4 fingers — one activated by the distal phalanx and the other by the middle phalanx. This initially sounds like it would be difficult to press 2 different buttons with 1 finger, but I think the way they’re positioned would make it actually work really well.

Septambic Keyer

The septambic keyer is another DIY concept, hand-built and used by wearable computing folks. It’s fitted to your palm, and you squeeze your fingers toward a fist to activate the buttons. There’s one button for each of the 4 fingers, and 3 for the thumb.

Clove 2

The Clove 2 is a Bluetooth-enabled glove with 5 buttons — 1 for each finger. It’s another do-it-yourself project.

Software Options

I found an interesting project called ASETNIOP that allows chording on a standard Qwerty keyboard. (Assuming the keyboard supports multi-key roll-over well enough. Most keyboards should work.) It’s got a really nice online interactive tutorial. You basically chord with the 8 home-row keys, plus the spacebar.

For Android, a program called GKOS for Android uses multi-touch with 1 or 2 button presses. The buttons are different sizes, depending on their frequency. I like this concept, but I don’t think the implementation is quite ready for production.

I also found an app for iOS called Nintype that is a combination between Swype and a chording keyer.

Conclusion

I think I’m going to try the Twiddler. If I find the concept to be a good one, hopefully I’ll build myself a Chordite (or have someone build one for me).

I’m also going to play with implementing some of the ideas from ASETNIOP, using KeyRemap4MacBook.