Lessons learned from studying Fizzy test suite

37signals recently open sourced their latest application called Fizzy. Not surpringly it's a pretty standard Rails application, but I was curious if we can find something interesting in its test suite. Let's have a closer look.

Minitest Fixtures
Lessons learned from studying Fizzy test suite

Fizzy

The Fizzy application is a new play on Kanban board and project management. It has a comprehensive test suite focused mostly on unit and integration tests, althought other types of test are also present. The test suite is well-organized and follows Rails conventions with some application-specific patterns for multi-tenancy and UUID handling. Can we find more than that?

Test suite

Here's the bird eye view on the test directory:

test/
├── application_system_test_case.rb # System test configuration
├── test_helper.rb  # Main test configuration
├── test_helpers/  # 8 custom test helper modules
├── fixtures/  # 32 fixture files with test data
├── models/  # 74 model test files
├── controllers/  # 52 controller test files  
├── system/  # 1 system test file
├── channels/  # 1 channel test
├── jobs/  # 1 job test
├── mailers/  # 8 mailer tests
├── helpers/  # 4 helper tests
└── lib/  # 4 lib tests

Having a single system test might be surprising at first, but not if you follow DHH's latest takes on testing. System tests are simply used as simple smoke tests and in fact the test file is just called like that, smoke_test.rb .

Findings

Multi-tenant setup

The test suite handles Fizzy's URL-based multi-tenancy elegantly by setting selected tenant as Current.account :

# test/test_helper.rb
setup do
  Current.account = accounts("37s")
end

This goes hand to hand with setting default_url_options :

# Integration test setup
setup do
  self.default_url_options[:script_name] = "/#{ActiveRecord::FixtureSet.identify("37signals")}"
end

Setting a script name is important because AccountSlug::Extractor extracts the tenant name out of the URL so the application thinks it's being mounted at / . I am still thinking if I should bring this to my Rails template.

Session management

Fizzy uses Indentity model to authenticate users and these identities can then be connected with various users of User model. For this there is a typical sign in helper called sign_in_as :

def sign_in_as(identity)
  cookies.delete :session_token

  if identity.is_a?(User)
    user = identity
    identity = user.identity
    raise "User #{user.name} (#{user.id}) doesn't have an associated identity" unless identity
  elsif !identity.is_a?(Identity)
    identity = identities(identity)
  end

  identity.send_magic_link
  magic_link = identity.magic_links.order(id: :desc).first

  untenanted do
    post session_magic_link_url, params: { code: magic_link.code }
  end

  assert_response :redirect, "Posting the Magic Link code should grant access"

  cookie = cookies.get_cookie "session_token"
  assert_not_nil cookie, "Expected session_token cookie to be set after sign in"
end

It's pretty standard stuff. We can notice we still have assertions here to fail early and we can also see the untenanted block. This is necessary because as we just covered, Fizzy assumes a tenant for the URLs.

These identities use magic links. To test magic links we create a link for the identity and pass it to session_magic_link_url :

# test/controllers/sessions/magic_links_controller_test.rb
test "create with sign in code" do
  identity = identities(:kevin)
  magic_link = MagicLink.create!(identity: identity)

  untenanted do
    post session_magic_link_url, params: { code: magic_link.code }

    assert_response :redirect
    assert cookies[:session_token].present?
    assert_redirected_to landing_path, "Should redirect to after authentication path"
    assert_not MagicLink.exists?(magic_link.id), "The magic link should be consumed"
  end
end

The user (or more specifically an identity) gets a cookie and the magic link should be no more.

Custom UUID generation

Fizzy supports both SQLite and MySQL and makes them compatible by using UUIDv7. The test suite includes a sophisticated UUID generation system for fixtures that maintains deterministic ordering:

def generate_fixture_uuid(label)
  # Generate deterministic UUIDv7 for fixtures that sorts by fixture ID
  # This allows .first/.last to work as expected in tests
  # Use the same CRC32 algorithm as Rails' default fixture ID generation
  # so that UUIDs sort in the same order as integer IDs
  fixture_int = Zlib.crc32("fixtures/#{label}") % (2**30 - 1)

  # Translate the deterministic order into times in the past, so that records
  # created during test runs are also always newer than the fixtures.
  base_time = Time.utc(2024, 1, 1, 0, 0, 0)
  timestamp = base_time + (fixture_int / 1000.0)

  uuid_v7_with_timestamp(timestamp, label)
end

This ensures that fixtures have predictable UUIDs that sort correctly, test-created records are always newer than fixtures and .first /.last work as expected.

This is of course the just the generation itself, but to take advantage of it they define the identify method:

module FixturesTestHelper
  extend ActiveSupport::Concern

  class_methods do
    def identify(label, column_type = :integer)
      if label.to_s.end_with?("_uuid")
        column_type = :uuid
        label = label.to_s.delete_suffix("_uuid")
      end

      # Rails passes :string for varchar columns, so handle both :uuid and :string
      return super(label, column_type) unless column_type.in?([ :uuid, :string ])
      generate_fixture_uuid(label)
    end

    # ...
end

ActiveSupport.on_load(:active_record_fixture_set) do
  prepend(FixturesTestHelper)
end

Once that's in place we can generate the UUIDs for fixtures:

logo_3:
  id: <%= ActiveRecord::FixtureSet.identify("logo_3", :uuid) %>
  card: logo_uuid
  creator: system_uuid
  created_at: <%= 1.day.ago %>
  account: 37s_uuid

External APIs

The suite uses VCR for recording HTTP interactions, set to filter sensitive data and come with a custom smart timestamp filtering:

VCR.configure do |config|
  config.allow_http_connections_when_no_cassette = true
  config.cassette_library_dir = "test/vcr_cassettes"
  config.hook_into :webmock
  config.filter_sensitive_data("<OPEN_AI_KEY>") { Rails.application.credentials.openai_api_key || ENV["OPEN_AI_API_KEY"] }
  config.default_cassette_options = {
    match_requests_on: [ :method, :uri, :body ]
  }

  # Ignore timestamps in request bodies
  config.before_record do |i|
    if i.request&.body
      i.request.body.gsub!(/\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2} UTC/, "<TIME>")
    end
  end

  config.register_request_matcher :body_without_times do |r1, r2|
    b1 = (r1.body || "").gsub(/\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2} UTC/, "<TIME>")
    b2 = (r2.body || "").gsub(/\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2} UTC/, "<TIME>")
    b1 == b2
  end

  config.default_cassette_options = {
    match_requests_on: [ :method, :uri, :body_without_times ]
  }
end

This is pretty cool because VRC cassettes remain reusable across different test runs and don't break due to timestamp mismatches.

Activity spike detection

Fizzy comes with tests for detecting unusual card activity patterns:

test "concurrent spike creation should not create multiple spikes for a card" do
  multiple_people_comment_on(@card)
  @card.activity_spike&.destroy

  5.times.map do
    Thread.new do
      ActiveRecord::Base.connection_pool.with_connection do
        Card.find(@card.id).detect_activity_spikes
      end
    end
  end.each(&:join)

  assert_equal 1, Card::ActivitySpike.where(card: @card).count
end

Here's the code for the helper:

def multiple_people_comment_on(card, times: 4, people: users(:david, :kevin, :jz))
  perform_enqueued_jobs only: Card::ActivitySpike::DetectionJob do
    times.times do |index|
      creator = people[index % people.size]
      card.comments.create!(body: "Comment number #{index}", creator: creator)
      travel 1.second
    end
  end
end

This is a nice show of using threading while getting a dedicated connection from the pool for each.

There is a special clear_search_records helper for testing full-text search which has to work both for SQLite and MySQL. We can see how we can run a query for each MySQL shard where we need it:

def clear_search_records
  if ActiveRecord::Base.connection.adapter_name == "SQLite"
    ActiveRecord::Base.connection.execute("DELETE FROM search_records")
    ActiveRecord::Base.connection.execute("DELETE FROM search_records_fts")
  else
    Search::Record::Trilogy::SHARD_COUNT.times do |shard_id|
      ActiveRecord::Base.connection.execute("DELETE FROM search_records_#{shard_id}")
    end
  end
end

Fin

So that's about it, just a few wisdom nuggets from Fizzy. Hopefully it shows you that test suites and be small, elegant, and fast if you stay on the Rails default testing stack.