Website Checklist

While creating a web page isn’t too difficult, there are a lot of moving parts involved in creating an effective website. We’ve written up this checklist as a guide for anyone who has decided that they need a website.

This list is for a basic static website. A web application will require significantly more effort. We’re working on a separate checklist for web apps.

Overview

  • Determine your goals
  • Develop a business plan
  • Register a domain name
  • Set up DNS hosting
  • Set up web hosting
  • Design and develop the site
  • Deploy and test the site
  • Market your site
  • Set up additional services
  • Maintenance

Goals

  • What do you want to achieve from having the site?
  • What “call to action” do you want visitors to take?
  • How will you measure success?
  • What will be the focus of the site?
    • Info (“brochure”) for an existing business
    • Blogging
    • Sales
    • Community
    • Web app
    • Mobile app

Business Plan

  • Who is your target audience?
  • Marketing
    • How will you get them to come to your site?
    • How will you get them to buy something?
  • Who is your competition?
    • How do you compare to them?
    • How do you differentiate from them?
    • What is your niche?
    • What makes your site better than theirs?
  • How will you make money?
    • Bringing people into brick and mortar business
    • Advertising
    • Periodic billing/subscriptions
    • Selling goods/services
    • Get bought out
  • Pricing
    • Aim high — it’s easier to lower prices than raise them
    • Tiered pricing often makes sense; most people pick the middle tier
  • What will it take to have a positive ROI?

Domain Name

You’ll probably want your own domain name.

  • Think of a name
    • Stick with a .com name if possible; don’t use .biz
    • Some other top-level domains come into fashion on occasion
      • .io is pretty popular right now
  • Check availability of the name
  • Keep checking names, until you find a good name that is available
  • Register the name with a respectable registrar
    • DNS Registrars run from $10-$35/yr
    • DO NOT allow your web host provider to own/control your name
    • You may want to grab the .net, .org, and other versions of the same name
    • Multiple-year registration is cheaper, but it’s easier to forget how to renew it
  • Your registrar will likely point your domain to an “under construction” page initially
  • DO NOT LOSE YOUR NAME!
    • Spammers and pornographers will take it over if your registration lapses
    • Make sure your contact info (especially email address) is up-to-date
  • Beware scams (usually by US mail) trying to get you to renew with a different registrar
  • Note that the email address you register with will get spammed
    • Some registrars provide some protection for a fee

DNS Hosting

You’ll need to have someone take care of the servers that tell the world what server addresses your domain name corresponds to.

  • Find a DNS hosting provider
    • We like DNSimple; they also do domain registration
  • Provide the name servers to your DNS registrar

Web Hosting

You’ll need servers to host your web site on. There are a lot of options available, from virtual hosts (a Linux server where you control everything) to application-specific services.

  • Talk to your developer or designer first!
    • Web host can significantly restrain the development environment
  • Cost
    • $10 to $500 / month is typical
  • Bandwidth
    • Number of users × how often they use it × average “page” size
    • What happens if you go over?
  • Up-time
    • What kind of down-time can the site sustain?
    • Higher guaranteed uptime costs more
    • What if the specified uptime is not met?
  • Development environment
    • What programming languages are installed?
    • What databases are installed?
    • What libraries are installed?
    • What if other libraries are required?
  • Shared/dedicated/virtual hosting
    • Shared means others are using the same machine, with security implications
    • Dedicated is expensive, but you “own” the whole machine
    • Virtual is somewhere in between
      • You have a “slice” of a machine dedicated to your site
  • How responsive is the host to problems and requests?
  • Backups
    • What do they back up?
    • How often do they back up?
    • How can files be restored?

Design and Development

Designing and developing the site can vary from picking an existing template and adding content, to developing a full web application.

  • Cost ($30 – $300 / hr)
  • Project management
    • Story/task management
  • Revision control
    • Ensures changes can be rolled back quickly
  • Functionality
    • What does the site need to do?
  • Usability
    • How easy is it to use each page?
    • Is it easy to navigate the site to find what you’re looking for?
    • Check for broken links
    • Check for ADA/508 compliance
    • Spell checking and grammar checking

Deploying and Testing

  • Can updates be deployed quickly?
    • Deploy early and often, so it’s not such a big deal, and it becomes routine
  • Consider a staging and/or beta site
    • Test everything thoroughly on staging site before deploying to production
    • Staging site should (mostly) use the same code as production
    • Staging/test site should not process credit cards, etc.
  • Automated testing
    • Prevents regressions
  • Exploratory testing
    • See how things work
    • See if you can break things
  • Security testing
    • Penetration testing
  • Performance
    • Load testing
  • Beta testers

Marketing

  • Search engine “optimization” (SEO)
    • Good design, good URLs, and well-written HTML should cover most of this
    • Submit site to search engines
    • Use robots.txt and site maps
  • Directories
  • PR sites
  • Targeted groups
  • DO NOT send out spam

Additional Services

What other services do you need, besides just the web site?

  • Email
  • Blog
  • Wiki
  • File storage
  • Forums
  • Authentication
  • Customer interaction
    • CRM
    • Feedback
    • Bug tracking
    • Customer Q&A (StackExchange)
    • Follow-up emails to customers to offer assistance

Maintenance

Over the lifetime of the site, you’ll likely pay more in maintenance costs than the upfront costs.

  • Responding to user emails
  • Requests for info
  • Feedback about the site
  • Password resets?
  • Tracking bug reports and feature requests
  • Site improvements
  • Additional content
  • Moderation of user content
    • Spam removal
  • Log analysis
    • Google Analytics
  • Assessing advertising effectiveness
  • Analysis of revenues/profitability
  • Upgrades
    • New/improved functionality
    • Bug fixes
    • Upgraded infrastructure
  • Down-time
    • Web host
    • Upgrades
    • Accidents
    • Bugs
  • Backups
    • Testing restoring from backups
  • Payments for services
    • Domain name registration – DO NOT LOSE YOUR NAME!
    • Web hosting
    • Marketing/advertising

 

Not Quite Callbacks

I’ve been working on application architectures based on Uncle Bob’s Ruby Midwest talk, following the hexagonal architectural pattern. I posted an article a couple months ago showing a way that works fairly well in Rails, and some accompanying Rails example code. But there was one thing I wasn’t quite happy with.

The problem is that we used callbacks (actually, a publish/subscribe mechanism) in a situation where they don’t seem to quite fit:

  def show
    interactor.on(:display) { |order| render order }
    interactor.on(:not_found) { |order_id| render status: 404 }
    interactor.get(params[:id])
  end

What we really want is to respond in different ways, depending on the result of the call to interactor.get(). There’s no good reason to define the responses before the call. It makes a lot more sense to define the responses after the call, because they’ll happen after the call. I’d much prefer that the code be written in the order that it will be run.

I discussed this problem with my friend and colleague, Amos King. We came up with a better solution, which puts things back in the right order:

  def show
    interactor.get(params[:id]) do |on|
      on.display { |order| render order }
      on.not_found { |order_id| render status: 404 }
    end
  end

He even wrote a small library to do this, which he called Riposte. I’m not sure what to call this pattern, but it seems to work pretty well in this situation. I suppose that they’re still technically callbacks, because they’re passed in in the block that’s passed in to the call to interactor.get(). But due to the magic of Ruby blocks, we get to put them in the order they should be.

Riposte also gives you the option of using the response object directly, instead of passing a block:

  def show
    on = interactor.get(params[:id])
    on.display { |order| render order }
    on.not_found { |order_id| render status: 404 }
  end

This shows that it’s just returning an object, with the twist that the response object has methods that take blocks. The nested blocks variant is really the same thing, except that it’s yielding to the response object instead of returning it.

I’ve decide that is the pattern I’d like to use for interactions and their callers within Ruby hexagonal architecture.

Architectural Thoughts

I’ve started working on my own framework in Ruby in the past couple days. It’s built upon my recent work at understanding Uncle Bob’s Ruby Midwest 2011 talk, and his article on Clean Architecture, and the resulting hexagonal architecture (AKA ports and adapters).

Somehow my research in that vein led me to Gary Bernhardt’s Boundaries talk. I’ve read a lot about the talk, and knew about the idea of “functional core / imperative shell”. And I’ve worked with a lot of similar ideas lately. But I believe this is the first time that I actually watched the whole video.

Even after having read a lot about similar ideas, it was pretty mind-expanding. Gary’s really good at presenting these kinds of ideas in a simple way.

OOP as usually taught includes encapsulation of data together with behavior, with mutable objects. Functional programming separates data and behavior, with mostly immutable data. From experience, encapsulating data and behavior together seems helpful. But experience also shows that immutability is useful. So it would be good to have both of those together. This is something I’ve been thinking for a few years — how best do we get both?

Gary calls the combination “FauxO”. Logic and data are still combined, but there’s no mutation. Anywhere OOP would normally have mutation would just generate a new object. There’s no language restriction involved in enforcing immutability — just discipline.

But without mutability, it’s hard to do IO and maintain state. So Gary’s solution is to encapsulate as much as possible into an immutable (functional or FauxO) core, and around that, use an imperative (traditional OOP) shell. The functional core contains the bulk of the logic, and the imperative shell is a glue layer that handles the real world, including disk, network, and other I/O.

The result of this is that the shell has fewer paths, but more dependencies. The core contains no dependencies, but encapsulates the different logic paths. So we’re encapsulating dependencies on one side, and business logic on the other side. Or put another way, the way to figure out the separation is by doing as much as you can without mutation, and then encapsulating the mutation separately.

I love how this naturally breaks things up, so that the core is all testable with unit tests, and the imperative shell is tested with integration tests. And since the shell has few or no logic paths, you get the testing pyramid, with more unit tests and fewer integration tests. The whole thing ends up being quite beautiful. Tests end up being very fast without any extra effort — not even stubbing or mocking. This tells us that things have been decomposed very well — an elegant design.

Gary makes the case that immutable objects can be treated as values, and passed across boundaries. Even process boundaries. This is something I’ve noticed as I’ve been working on my own Uncle Bob style hexagonal framework, but nobody in that camp ever mentioned that — they prefer DTOs or something more like hashes. I’m completely against hashes, because of the “stringly-typed” problem. And I don’t see much advantage in a DTO if I’ve got an immutable object; I’d be basically copying the object to an almost identical object. And I’d be losing any laziness possible for derived values within the original immutable object.

It’s striking to me how Gary’s image of an imperative shell around a functional core, plus Net, Disk, and State outside of the shell mirror’s Uncle Bob’s concentric circles. Uncle Bob has entities in the middle, surrounded by use cases, surrounded by Web, DB, and UI.

Another advantage that Gary shows is that breaking things up this way allows easy concurrency. In his example, he shows using the actor model — either just using threads and queues, or an actor library (or language feature).

After several years of thinking about the architectural issues seen in most large Rails apps, I’m starting to come to an understanding of how to combine all these ideas and come up with an architecture that will work better.

 

Hexagonal Rails Controllers

I’ve had a long love-hate relationship with Rails. I love the MVC framework and how it’s improved our speed of writing web apps. But I’ve never really been completely happy with it. I don’t generally agree with most of its opinions. I prefer models that follow the Data Mapper pattern, not the Active Record pattern. This includes separating the persistence layer from the models’ business logic. I prefer Slim or HAML to ERB. I prefer RSpec to Test::Unit or MiniTest. When Merb hit the scene, I was ready to make the jump, until Merb merged with Rails.

So inspired by PJ Hagerty’s recent article on alternative Ruby web frameworks, I started thinking about how I’d write a replacement for Rails. I’d definitely keep the basic MVC framework. But I’d also want to implement a more hexagonal architecture.

I started sketching out what this would look like, but I ended up starting with a Rails controller and finding the simplest way to make it hexagonal. I really don’t like callbacks, because they make tracing program execution difficult. But I didn’t see any other alternative. I found a simple pub/sub Ruby library called Wisper. It literally has only publish, subscribe, and on methods. (You use on to register single callbacks via blocks, and subscribe to register an object with method names corresponding to the callback names.)

The trick was figuring out how to break the controller into 2 pieces. What finally helped me was to find the single responsibilities of the 2 pieces. The Rails controller would remain in charge of managing the web interface, but would delegate to the other piece to handle any application-specific business logic. I decided to re-watch Uncle Bob Martin’s “Architecture The Lost Years” talk, which was the first time I was introduced to the ideas of Hexagonal Architecture. (He doesn’t name the architecture in the talk, but later calls it Clean Architecture.) He does a decent job of explaining how to break these 2 pieces apart. He used the term “interactor” in that talk, so I decided to go with that. He said that Jacobsen calls it a Control Object in Object Oriented Software Engineering, but that’s too close to Rails’s “controller”.

So here’s an example of what I ended up with:

class OrderController < ApplicationController
  def index
    interactor.on(:display) { |orders| render orders }
    interactor.list
  end

  def show
    interactor.on(:display) { |order| render order }
    interactor.on(:not_found) { |order_id| render status: 404 }
    interactor.get(params[:id])
  end

private

  def interactor
    @interactor ||= OrderInteractor.new
  end
end
require "wisper"
require "order"

class OrderInteractor
  include Wisper.publisher

  def list
    orders = Order.all
    publish(:display, orders)
  end

  def get(id)
    order = Order.find(id)
    publish(:display, order)
  rescue ActiveRecord::RecordNotFound
    publish(:not_found, id)
  end
end

I do have a few problems with this solution though. I’m not a fan of the name “interactor” for the business logic. I thought about calling it OrderOperator, or maybe OrderOperations, because it’s really a collection of operations. Perhaps it would be better to separate each operation into a separate class. Trailblazer does it that way. And for more complicated business logic, I would do that too, using the Method Object pattern. But like a Rails controller, there’s a lot in common among all the operations. I feel like a separate class for each operation for each would create too many coupled classes.

I’m also uncomfortable with the fact that the controller is delegating almost everything to the interactor. I guess this is OK, but it feels like there’s too little left when every line starts with interactor. I suppose extracting things some more would help mitigate this concern I’ll likely write a small gem to perform that extraction. I expect that that will allow a typical controller to be written in only a few lines. And maybe the same for the interactor side.

With the business logic extracted out of the controller, it was really easy for me to write a command-line version of the app. As Uncle Bob says, “the web is not particularly important to your application.”

I’ve put the code for this example on GitHub: https://github.com/boochtek/hexagonal-rails. I’ll likely experiment with it some more over the next few weeks and months.

TDD Is Alive And Well

I went to RailsConf this year, and the very first talk was a keynote by David Heinemeier Hansson (DHH), the creator of Ruby on Rails. The TL;DR of his talk was “TDD rarely has value”. He followed up with a blog post the next day, titled “TDD is dead. Long live testing.“, and 2 more posts. I think this line of thought is terribly misguided, and causing more harm than good. This article is my response.

First, I would like to address the good points of the talk. He said that programming is pseudoscience, and that people want to tell us that there’s a secret to being a better programmer. But what it really takes is working hard — reading a lot of code, writing a lot of code, and rewriting a lot of code. He’s right. And I also agree with him that you should forget about patterns for a while when learning to code. Beginners try to throw patterns at a problem instead of letting the patterns emerge where they’re supposed to.

I don’t completely agree that programming is a pseudoscience. In some ways it is, but I think it’s more of a craft. It’s a craft, because there’s a lot of science involved, but there’s also an art to doing it well. And like any craft, you’re always working to get better. So to respond to DHH’s stance that “software is more like poetry than physics”, I think it falls
somewhere in between.

With regard to the software engineering practices we use, there really isn’t much science available, mostly because it’s a soft science. That is, it’s really hard to isolate a single variable when comparing code between projects. And nobody has the time or money to write the same code so many times that the differences would be statistically significant.

So we don’t have much science on TDD. But we do have some. Here’s a collection of several: StudiesOfTestDrivenDevelopment. And here’s one that explicitly looks are the difference between test-first and test-last: Does Test-Driven Development Really Improve Software Design Quality? What do these tell us? They tell us that TDD costs us about 10-30% in short-term productivity; reduces bugs by 30-90%, and decreases code complexity by about 30%. As Code Complete tells us (in section 20.5, with studies to back it up), improving quality reduces development costs. So, like most Agile practices, this is a case where spending a bit more time in the short term leads to time savings in the long term.

The more important lesson in the talk was that you have to do what works best for you and your situation. If TDD doesn’t give better results, then either find out how to make it give better results, or stop using it. As we often say in the Agile world, Agile doesn’t mean that you can stop using your brain. While I think TDD is appropriate in most situations, there are cases where it’s not worth the additional up-front cost. If the most important thing for your project is time-to-market, then not testing might be the right decision for you.

To me, TDD provides a bunch of benefits. First and foremost, TDD is a design discipline. It ensures that I think about how my code will be used before I think about how to implement it. This is very powerful in ensuring that the code is well-written from the perspective of other code using it.

Tested code provides confidence to be able to make changes without breaking things. If we write tests after the code, we’re less likely to write them. Tests written after the code also tend to test the implementation instead of the desired functionality. What we really want is tests written as a specification. With tests as a specification, we can come back later and understand why code was written. Without tests, or with poor tests, we can’t understand why the code is there; if we want to rewrite it, we don’t have the confidence that we’re not missing something. Writing tests first also ensures that we only write the code that is needed to implement the required functionality.

I’m not sure why DHH hasn’t “gotten” TDD. I’m not sure if it’s because he’s  a better coder than average, or if he just thinks in a different way than most of us. I think it’s partly because he doesn’t understand TDD, which he admitted might be the case. And I think he’s conflating TDD and unit testing.

DHH is influential in the developer community, especially those newer to Ruby and Rails. People listen to what he has to say. I was happy to see almost every other speaker made fun of DHH’s ideas, and most of the crowd knew better. But there will be a lot of others who will hear DHH, respect his opinions, and not give TDD the try that it deserves. And that’s sad, because it will lead to an overall reduction in code quality in the world.

Here are some other people’s thoughts on the matter:

 

Includable ActiveRecord

I created a Ruby gem recently, called includable-activerecord. It’s pretty small, but I thought I might explain why I created it, and discuss its implementation.

Classical Inheritance

When you use ActiveRecord, you normally include it in your model like this:

class User < ActiveRecord::Base
  # ...
end

Your User class is inheriting from the ActiveRecord::Base class. This is class-based inheritance, also called “classical” inheritance. (That’s “classical” as in “class”, not as a synonym for “traditional”.) Class-based inheritance represents an “is-a” relationship. So we’re saying that a user is an ActiveRecord base. Another way to say this is that User is a subclass of ActiveRecord::Base.

There are a few problems with this. First, what is a “base”? The name was chosen because it’s a base class. But just like we don’t give factory classes names like UserFactory (at least not in Ruby), we shouldn’t name base classes Base.

I suppose that we’re trying to say that this is an ActiveRecord model. That sounds fine at first glance — this is our model for users. But what if we also want to say that a user is a person? Ruby doesn’t allow inheriting from multiple classes. Now we have to choose whether to inherit from ActiveRecord::Base or Person. Person makes more sense, because it fills the “is-a” rule better. Class inheritance is intended for a hierarchical “is-a” relationship, such as “a user is a person”, or “a circle is a shape”. But since ActiveRecord::Base is a base class, we have to use it as our base class.

We could work around this problem by subclassing Person from ActiveRecord::Base and then subclassing User from Person. That’s fine if Person is also a model that we store in the database. But if that’s not the case, then we have a problem.

Mixins

Ruby provides another way of implementing inheritance — mixins. We often don’t think of this as an inheritance model, but it really is. When we include a module, that module gets added to the class’s ancestor chain. We can mix in as many modules as we want.

Mixins indicate more of an “acts like” relationship than an “is-a” relationship. It’s for shared behavior between classes that don’t have a hierarchical relationship. For example, when we mix in the Enumerable module, we’re saying that we want our class to act like other classes that include Enumerable. That sounds more like what we want ActiveRecord to be. We want our user model to behave like other ActiveRecord models, in the way that they can persist to a database.

But ActiveRecord doesn’t support that. Almost all the other Ruby ORMs do; as we’ve shown above, this is for good reasons.

Implementation

So I decided to see if I could implement the equivalent of the ActiveRecord::Base class as a module that could be mixed into model classes. I decided to call my mixin module ActiveRecord::Model, because classes that mix it in will behave as ActiveRecord models.

It turns out that ActiveRecord::Base is a pretty complex class. It includes and extends a lot of other modules. Luckily, as of ActiveRecord 4.0, that’s all the code it includes.

The module only defines a single class method, included. This is one of Ruby’s many hook methods. It gets called when the module in question gets included in another module, and receives that other model as its argument. All we need to have this method do is to include everything that ActiveRecord::Base includes, and extend everything that ActiveRecord::Base extends. Ruby provides a method that’s defined on all classes, called included_modules, which we can use to get the list of everything that’s included in ActiveRecord::Base. Unfortunately, there’s no equivalent list of extended_modules. But a quick search on Stack Overflow found an implementation of extended_modules that we could use.

So with a bit of magic (i.e. hooks and meta-programming), we can get the lists of constituent modules from the ActiveRecord::Base class, and include them in our ActiveRecord::Model module.

So with all that, we can now include the includable-activerecord gem and mix it in, with all the advantages that provides:

class User
  include ActiveRecord::Model
  # ...
end

It was exciting to be able to make this work. Since I wrote it as a proof of concept, I haven’t written any tests yet. But it seems to be working just fine. The main thing I really need to look into is making sure that plugins that extend ActiveRecord::Base from their own code will still work. I’m pretty sure this will work out of the box, because the ActiveRecord::Model.included doesn’t run until the model class is loaded, and that happens after those plugins have initialized themselves.

Testing Rails Validators

It’s challenging to test Rails custom validators.

I recently had to write a validator to require that an entered date is before or after a specified date.

It didn’t seem like writing the validator would be too difficult – I’ve written custom validators before, and date comparisons aren’t all that tricky. But when it came time to write the tests, I ran into several issues. And since I always try to follow TDD / test-first, I was blocked before I even began.

The biggest issue was the ActiveModel::EachValidator#validates_each API. It’s definitely not a well-designed API. You write your validator as a subclass, overriding validates_each. The method takes a model object, the name of the attribute of the model being tested, and the value of that attribute. You can also get the options passed to the custom validator via the options method. To perform a validation, you have to update the model’s errors hash.

The big flaw in the API is that instead of returning a result, you have to update the model. This needlessly couples the model and the validator. And it violates the Single Responsibility Principle — it has to determine validity of the field, and it has to update the errors hash of the model. This is not conducive to testing. Testing this method requires testing that the side-effect has taken place in the collaborator (model), which means it’s not really a unit test any more.

So to make it easier to unit test the validator, I broke the coupling by breaking it into 2 pieces, one for each responsibility. I moved the responsibility for determining validity to a separate method, which I called errors_for. It returns a hash of the errors found. This simplified the validates_each method to simply take the result of errors_for and update the errors hash of the model:

def validate_each(record, attribute_name, attribute_value)
  record.errors[attribute_name].concat(errors_for(attribute_value, options))
end

This made it much easier to unit test the errors_for method. This method doesn’t even need to know about the model — only about the value of the attribute we’re trying to validate. We simply pass in the attribute’s value and the options.

So we could write the tests without even pulling in ActiveRecord or any models:

describe DateValidator do
  let(:validator) { DateValidator.new(attributes: :attribute_name) }
  let(:errors) { validator.errors_for(attribute_value, validation_options) }

  describe 'when attribute value is NOT a valid date' do
    let(:attribute_value) { 'not a valid date' }
    it { errors.must_include 'is not a valid date' }
  end

  describe 'when attribute value IS a valid date' do
    let(:attribute_value) { Date.parse('2013-12-11') }
    it { errors.must_be :empty? }
  end
end

And the errors_for method looked something like this:

def errors_for(attribute_value, options)
  unless attribute_value.is_a?(Date)
    return [options.fetch(:message, "is not a valid date")]
  end
  []
end

Integration testing can also be a bit of a challenge. I recommend following the example from this Stack Overflow answer. Basically, create a minimal model object that contains the field and the validation. Then test that the model behaves like you expect with different values and validations.