Kyle Keesling

Migrating from Sidekiq to Solid Queue

Wed, 03 Jan 2024 11:00:00 -0500

With the recent release of Solid Queue, and a little bit of extra free time due to the holidays, I made a decision to migrate my apps away from Sidekiq. The idea of simplifying my infrastructure requirements by removing Redis from the equation, and the relatively vanilla requirements I have for a background queuing system were just enough to convince me this was a worthwhile exercise.

I won’t bore you with every machinations of copying, pasting, and deleting code since it’s realtively straightforward, but I did think it was worth sharing a few highlights, considerations, and gotchas I came across.

The Migration

The switchover in my case was very straightforward since my use of Sidekiq was all via Active Job. It consisted of:

Swapping out the gems in my Gemfile
Running the Solid Queue installation scripts
Removing the Sidekiq UI mounting lines from routes.rb
Deleting sidekiq.rb from config/initializers
Tweaking my Solid Queue config

Configuration

I migrated two apps to Solid Queue. One of them mainly uses ActiveJob to send emails and run reports, so I simply replied on the default config, but my other app does utilize multiple queues, so I ended up with the following config for it:

# config/solid_queue.yml
default: &default
  workers:
    - queues: "*"
      polling_interval: 2
    - queues: real_time
      polling_interval: 0.1

development:
  <<: *default

test:
  <<: *default

production:
  <<: *default

My goal here was to create a priorty queue, called real_time, so I decided to add a second worker dedicated soley to those jobs, and to poll for them at a much higher frequency.

A Few Considerations

My app with the completely vanilla config deployed and worked without a hitch, but I did run into a few hiccups with my custom config.

DB Connection Pool

I deploy my apps on Heroku, and by default the app is setup to look at the DB_POOL environmental variable to set the connection pool in config/database.yml. This is fine for app instances but since I’m running a dedicated worker dyno for Solid Queue, some extra work was needed.

By default each Solid Queue worker has 5 threads, meaning that since my app has two workers, I needed a connection pool that allowed 10 connections. I was able to accomplish this by using a specfic enviroment variable, SOLID_QUEUE_DB_POOL, as is suggested in the Heroku docs.

# Procfile
web: bundle exec puma -C config/puma.rb
worker: DB_POOL=$SOLID_QUEUE_DB_POOL bundle exec rake solid_queue:start
release: bundle exec rails db:migrate

Error Handling

Note: A previous version of this post referred to the on_thread_error setting to catch job errors. After some more digging I realized this setting is not for job errors but rather actual errors that occur that are related to the thread directly.

By default the library silently handles errors and marks them as failed, but I prefer being proactively notified when something happens. To do this you’ll need to leverage an around_perform callback to wrap your jobs in an error handler, then report them as necessary

 class ApplicationJob < ActiveJob::Base
  around_perform do |job, block|
    capture_and_record_errors(job, block)
  end

  def capture_and_record_errors(job, block)
    block.call
  # I had to use rescue here instead of a `Rails.error` block because Honeybadger ignores the `Rails.error.report` call
  # in favor of their own error handler, which is fine in most cases, but unfortunately doesn't work here. Report would be
  # great here because it re-raises the error, but instead I have to do that manually
  rescue Exception => e
    Rails.error.set_context(**error_context(job))
    Rails.error.report(e)
    raise e
  end

  def error_context(job)
    {
      active_job: job.class.name,
      arguments: job.arguments,
      scheduled_at: job.scheduled_at,
      job_id: job.job_id
    }
  end
end

A huge thanks goes out to Rosa Gutierrez for not only her wonderful work on this gem, but also for taking the time to respond to Github issues.

Failures and Retries

By default, Sidekiq has a built-in retry mechanism, but this is not the case for Solid Queue. If automatic retrying is important for you you’ll need configure this on a per-job basis, or in application_job.rb, via Active Job’s retry_on API.

Wrapping Up

So far I’m feeling good about the simplification that this has afforded me/my apps, but I do find myself missing the UI that Sidekiq provided, but it does not seem that we’ll have to wait too long for an answer for that either.

Deploying a Rails App Using Kamal and SQLite

Fri, 13 Oct 2023 00:00:00 -0400

There’s been a lot of excitement lately in the Rails community. Rails World just wrapped up, which brought with it some really great annoucements for the future of the platform, as well as the release of Rails 7.1.

This most recent release had a few particular additions/improvements that really piqued my interest in regards to how I approach deployment of new Rails apps, specifically around the inclusion of a standard Dockerfile as well as a focus on making SQLite a first-class citizen/consideration for production apps.

When you combine that with my recent discovery of the litestack gem, the recent release of Kamal, and the recent GoRails series on building your own URL shortner, and the stars had aligned.

Prerequisites

Before we continue, here are a few things you’ll want to make sure you’ve got setup/configured properly:

Your app is using Rails 7.1 and has the default Dockerfile available to it
Docker is installed on your local computer
You have an account with a hosting provider like DigitalOcean or Hetzner

SQLite + Litestack

For this project I wanted to host my project on the cheapest DigitalOcean droplet possible, but it included some advanced requirements/features, including ActiveJob and ActionCable for background tasks and Turbo Broadcasts. While this certainly could have been done in the more traditional way of installing a traditional database like PostgreSQL or MySQL and redis on the server, my goal was simplicity.

In contrast, leveraging a single technology like SQLite for all of these core aspects of the app would allow me to greatly simplify the hosting requirements, which is where Litestack comes in.

If you aren’t familiar, Litestack is a wonderful project with the following objective:

Litestack is a Ruby gem that provides both Ruby and Ruby on Rails applications an all-in-one solution for web application data infrastructure. It exploits the power and embeddedness of SQLite to deliver a full-fledged SQL database, a fast cache , a robust job queue, a reliable message broker, a full text search engine and a metrics platform all in a single package.

Installing the gem is very straightforward, drop it in your Gemfile and run the installation command rails generate litestack:install.

Kamal

I’ll admit it, I’m late to the Docker party, so there were many of the concepts I was (and am still) unfamiliar with, but the abstraction Kamal provides and the default Rails Dockerfile are just enough to allow me to get the job done.

That being said I do think there’s value in more broadly understanding Docker as well as things like Traefik, which is something that’s mentioned many times in Kamal’s docs, but is never really explained. I can piece together that it’s an orchestration and monitoring layer for your Docker containers, but I’ve found it difficult to learn much more about it and how to use it, so if you know of any good resources on better understanding it, please share.

The main gotcha I ran into was around how to store the SQLite databases in persistant storage. I solved this by configuring a Kamal volume and adding the following environmental variable to my .env file: LITESTACK_DATA_PATH=./storage.

After a decent amount of trial and error, and losing my database between deploys more than once 😆, the following deployment config is what ended up getting the job done.

# deploy.yml
service: url-shortner

image: kylekeesling/url-shortner

volumes:
  - "storage:/rails/storage"

servers:
  web:
    hosts:
      - 192.241.138.163
    options:
      "add-host": host.docker.internal:host-gateway
    labels:
      traefik.http.routers.rails_recipes.entrypoints: websecure
      traefik.http.routers.rails_recipes.rule: Host(`links.passtesting.com`)
      traefik.http.routers.rails_recipes.tls.certresolver: letsencrypt

registry:
  username: kylekeesling
  password:
    - KAMAL_REGISTRY_PASSWORD

env:
  clear:
  secret:
    - RAILS_MASTER_KEY
    - LITESTACK_DATA_PATH

traefik:
  options:
    publish:
      - "443:443"
    volume:
      - "/letsencrypt/acme.json:/letsencrypt/acme.json"
  args:
    entryPoints.web.address: ":80"
    entryPoints.websecure.address: ":443"
    entryPoints.web.http.redirections.entryPoint.to: websecure
    entryPoints.web.http.redirections.entryPoint.scheme: https
    entryPoints.web.http.redirections.entrypoint.permanent: true
    entrypoints.websecure.http.tls: true
    entrypoints.websecure.http.tls.domains[0].main: "links.passtesting.com"
    certificatesResolvers.letsencrypt.acme.email: "kyle.keesling@passtesting.com"
    certificatesResolvers.letsencrypt.acme.storage: "/letsencrypt/acme.json"
    certificatesResolvers.letsencrypt.acme.httpchallenge: true
    certificatesResolvers.letsencrypt.acme.httpchallenge.entrypoint: web

Credit and Thanks

I’d like to send out a huge thanks to Greg Molnar and Erik Minkel, who both provided some great content to help me along the way.

Testing Stripe Webhooks with Minitest

Fri, 29 Sep 2023 12:00:00 -0400

As I was looking to sure up some billing code in one of my projects, I figured it was finally time that I created some tests for the various Stripe webhooks that we used.

I was a bit dishearted that the only webhook testing examples I could find online pertained to RSpec, so with that I rolled up my sleaves and dug into the internals of both the stripe_event gem as well as Stripe’s offical gem.

After a few rounds of trial and error I was able to discern how Stripe generates the signature for webhooks and how I could mimic it to test my endpoint. You can see that below in the webhook_header method.

You’ll also need to grab sample payloads for each of the webhook events you’d like to test and save them as a JSON file in test/fixtures/stripe/#{event}.json, as seen in the webhook_payload method.

If you aren’t sure where to grab sample payloads from, the Stripe CLI has a trigger command that can make gathering them much easier.

The only other varaible you may have is the path to your webhook endpoint, which in my case is stripe_event_path, as seen in the deliver_payload method.

Take all of that and you get the following test case can be used as a base class for any tests you may need to write:

# test/stripe_test_case.rb
require "test_helper"

class StripeWebhookTestCase < ActionDispatch::IntegrationTest
  SIGNING_SECRET = ENV["STRIPE_SIGNING_SECRET"]

  def deliver_payload(event:)
    params, headers = webhook_setup(event)

    post stripe_event_path, params:, headers:
  end

  def webhook_setup(event)
    payload = webhook_payload(event)
    [payload, webhook_header(payload:)]
  end

  def webhook_payload(event)
    File.read(Rails.root.join("test", "fixtures", "stripe", "#{event}.json"))
  end

  def webhook_header(payload:)
    time = Time.now
    signature = Stripe::Webhook::Signature.compute_signature(time, payload, SIGNING_SECRET)
    header = Stripe::Webhook::Signature.generate_header(time, signature)

    {"Stripe-Signature": header}
  end
end

Once you have this test case, here’s an example test that could be used:

# test/webhooks/stripe/general_webhook_test.rb
require "stripe_webhook_test_case"

class Stripe::GeneralWebhookTest < StripeWebhookTestCase
  test "webhook endpoint returns proper status code" do
    deliver_payload(event: "charge.succeeded")

    assert_response :ok
  end

  test "webhook endpoint returns proper status code when no headers are provided" do
    post stripe_event_path, params: webhook_payload("charge.succeeded")

    assert_response :bad_request
  end

  test "webhook endpoint returns proper status code when improper headers are provided" do
    post stripe_event_path, params: webhook_payload("charge.succeeded"),
      headers: {"Stripe-Signature": "#{Time.now.to_i},v1=badsignature"}

    assert_response :bad_request
  end
end

In this project I also happen to be using the stripe_event gem, so a good deal of the actual route/endpoint logic is abstracted from my app. I’m currently considering rolling my own completely and removing this gem as a dependancy, as I’ve done in another project already, but that’s a topic for another post. The Stripe docs also have a really good example one could use as a starting point.

Add Stripe Checkout to your Rails app in Under 10 Minutes

Mon, 24 Jan 2022 13:45:00 -0500

Migrating to jsbundling-rails

Sun, 23 Jan 2022 11:00:00 -0500

Now that Webpacker is riding off into the sunset, many of us find ourselves switching over to using one of the solutions offered by jsbuilding-rails.

With the move comes a much simplier javascript story, which is a welcome change for many of us, but making the move can quickly lead to a few gotchas. Below I’ll outline the few that I’ve run into and how you can avoid them so you can ensure a smooth transition to using esbuild.

Gotcha #1 - Importing CSS in your Javascript

This has been one of the most common problems I’ve noticed folks having. Say you’re using a javascript package that also includes CSS assets, esbuild does allow you to import those, but will name the output file the same name as the root javascript file in app/javascript which in most cases will be application.js.

The result of that being two files output to app/assets/build , application.js and application.css - and that’s where the hang-up occurs. If you are also using cssbundling-rails it will by default name it’s output the same thing - application.css, causing a naming collision that will inevitably cause you to bang your head against the wall if you aren’t sure what’s going on.

There are two solutions here, one is renaming the output of your cssbundling-rails file, or renaming your app/javascript/application.js to something else, which in my case is what I did, making it application-esbuild.js. Now, esbuild will process your javascript and CSS and place two files named application-esbuild.js and application-esbuild.css into app/assets/build. Then all you need to do is make sure you include those files in the head of your application.html.erb file:

<!-- app/views/layout/application.html.erb -->

<%= stylesheet_link_tag "application", media: :all, data: { turbo_track: "reload" } %>

<%= stylesheet_link_tag "application-esbuild", media: :all, data: { turbo_track: "reload" } %>
<%= javascript_include_tag "application-esbuild", data: { turbo_track: "reload" } %>

Gotcha #2 - Having Two Javascript Directories

If you’re not starting fresh in Rails 7 there’s good chance you’ve got files in app/assets/javascript. If so you’ll also want to make sure you’ve got sprockets setup to serve those assets. This can be done in app/assets/config/manifest.js, just make sure you’ve got the following line in the config:

// app/assets/config/manifest.js

//= link_tree ../images
//= link_tree ../fonts
//= link_tree ../builds

// this is the key addition, referring to the file app/assets/javascript/application.js
//= link application.js

Of course you could also use link_tree or any of the other methods that sprockets provides in order to properly include the necessary javascript files in app/assets/javascript/, but I think explicitly including just one file helps keep things clean and easy to understand.

Also note that doing this also assumes that you’ve renamed your esbuild output file to application-esbuild.js as mentioned above, otherwise you’d run into similar naming collision issues that Gotcha #1 covers.

Now with Bridgetown

Sun, 23 Jan 2022 08:57:33 -0500

After over 9 years of running this site on Jekyll I finally made the jump to Bridgetown. Shoutout to Jared and the team for making this migration fairly easy and straightforward.

Setting Up Proper Amazon S3 Permissions for ActiveStorage

Tue, 28 Jan 2020 12:43:00 -0500

If you’ve found yourself marveling at how cryptic and impenetrable understanding AWS services, then you are definitely not alone. For much too long have I relied on doing a quick web search and blindly copying and pasting settings and policies, so when I found myself doing it again this week for one of my applications I decided it was time to slow down and actually understand what it was that I was doing.

The Problem

I’m creating a demo environment that needs to be completely separate from our production data, so part of that entails creating its own S3 bucket for use with ActiveStorage. Having not really done this since ActiveStorage was first released I begin DuckDuckGoing (is this even a thing?) for a how-to.

Some perfectly fine write-ups and tutorials popped up on the first page, but I noticed that all of them recommended setting up an IAM User with AmazonS3FullAccess permissions. As soon as I looked at the description for this policy it threw up a big red flag:

AmazonS3FullAccess: Provides full access to all buckets via the AWS Management Console

Full access to all buckets? That’s gonna be a big nope from me.

The Solution

At an absolute minimum the permissions need to be locked down to a single bucket, but ideally we’d only give the app the ability to perform solely the actions that are required for the framework to function properly. Luckily there’s a callout right in the Rails guide that states:

The core features of Active Storage require the following permissions: s3:ListBucket, s3:PutObject, s3:GetObject, and s3:DeleteObject. If you have additional upload options configured such as setting ACLs then additional permissions may be required.

Assuming you already have a bucket created, we just need to create an IAM User so we can generate the access_key_id and secret_access_key necessary to configure the config/storage.yml Amazon service.

Once you have your IAM user created you can use the policy template I’ve built down below as a guide to either create and attach the policy to a group you add your IAM user to, or just attach the policy directly to the user, which is what I did.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "VisualEditor0",
      "Effect": "Allow",
      "Action": [
          "s3:PutObject",
          "s3:GetObject",
          "s3:DeleteObject"
      ],
      "Resource": [
          "arn:aws:s3:::BUCKET_NAME_GOES_HERE/*"
      ]
    },
    {
      "Sid": "VisualEditor1",
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::BUCKET_NAME_GOES_HERE"
    }
  ]
}

Final Notes

To their credit, one or two articles said something to the extent of “you may want to do this differently in production”, but let’s be honest, most people don’t read the entire article and/or will not remember to do this, so I find it to be a much better idea to set things up right from the start.

Hopefully this helps you in your ActiveStorage-related endeavors, and if you have any suggestions or comments on how you handle S3 permissions, I’d love to hear them!

Getting Honeybadger Uptime Alerts via SMS with Zapier

Wed, 04 Sep 2019 05:00:00 -0400

UPDATE: Shortly after posting this Honeybadger was kind enough to let me know that this is functionality that they offer directly 😁

After using Honeybadger Uptime Monitoring for a few months, I found that these updates were sometimes getting lost in the everyday chatter of my error messages that I have posted to a Basecamp chat via Zapier.

With that in mind I came up with the idea to create a new Honeybadger webhook alert that would post only uptime updates to Zapier.

Below is an overview of the Zap I came up with:

With that now in place I now have Zapier text me whenever something big happens… which hopefully is rarely to never :)

Piping Your Honeybadger Events to Basecamp with Zapier

Wed, 03 Jul 2019 09:30:00 -0400

My team has been working on consolidating and improving our communication tools and practices, and as part of that we’ve decided to ween ourselves off of Slack and totally leverage Basecamp for all internal and customer communications.

One of the major things that the team appriciated in Slack was the #dev channel I had created that piped all of our deploys, pull requests, and errors into one central location, allowing everyone to know what’s going on without having to tap someone on the shoulder.

Basecamp provides an easy way, using a chatbot, to feed Github events into a Campfire chat, but getting our application errors in wasn’t quite as straightforward.

We use Honeybadger to track our errors, and while they offer many integrations, Basecamp isn’t one of them — enter Zapier.

How’d I do it?

Zapier made it a very simple process:

create a catch webhook to catch incoming webhooks from Honeybadger
create a POST webhook action that massages the JSON payload from Honeybadger, into the format Basecamp expects for a chatbot

The Result in Basecamp

Using the details and summary elements gives you the ability in Basecamp to create a collapsable post. In the sample above notice the little black right arrow, if you click it you can even get a little sample of the stacktrace. Here’s the value of the contents key I used for the chatbot payload:

    <details>
      <summary>
        <strong>💥({xxx__fault__id}){xxx__message}</strong><br>
        <a href="{xxx__fault__url}">View in Honeybadger</a>
      </summary>
      <br><hr><br>
      <pre>{xxx__notice__application_trace}</pre>
    </details>

And here’s a preview of what the expanded preview looks like:

Summary

We are still in the process of fulling moving over to Basecamp, but so far I’ve not missed Slack nearly as much as I thought I would.

In all honestly the things we’ve missed the most from Slack are the GIPHY integration, and the quick access to screen collaboration tools, so if you have any suggestions on how your team addresses those areas, I’d love to hear about it.

An Easier Way to Accommodate Deletion of ActiveStorage Attachments

Mon, 03 Dec 2018 17:45:00 -0500

A Little Background

As I continue to phase out Paperclip in favor of ActiveStorage, I’ve wanted to keep the methods I used to manage these assets as succinct and reusable as possible.

ActiveStorage gives us a dead simple way to save and update assets, but the ability to delete assets independently of the parent record, particularly if you’re using has_many_attached, has been left to each individual app to figure out.

I have stumbled upon a couple of different takes on how to potentially do this, but I’ve not been satisfied with their approaches. Many do not take into account authorization and permissions, which if you’re not careful, allows any user to delete any attachment they choose just by randomly hitting URLs in your application. With that in mind I set out to try and roll my own.

The Approach

Since all attachments are saved as the same object/models types, Attachment and Blob, we can use a common controller to interact and modify with them, regardless of the parent record that it belongs to.

In my case I chose to stick closely to the naming conventions already given to us, so I created a new controller named ActiveStorage::AttachmentsController, and since for now I’m only worried about deleting attachments, we only need one method, delete.

I use Devise for authentication, so we need to make sure that the user is logged in before we let them do anything, which we do with :authenticate_user!.

I also want to make sure that the user has the permissions to modify these attachments, so I check their permissions to see if they can modify the parent record using the authorize_attachment_parent! method. I use CanCanCan in this case, but you can always swap out your auth call to fit whatever method you use. This piece is critical to ensure that your users aren’t deleting anything they shouldn’t.

The purge_later call will take care of the actual file deletion in your background queue, and will delete the corresponding Blob record.

    # app/controllers/active_storage/attachments_controller.rb
    class ActiveStorage::AttachmentsController < ApplicationController
      before_action :authenticate_user!
      before_action :set_attachment, :authorize_attachment_parent!

      def destroy
        @attachment.purge_later
      end

      private

        def set_attachment
          @attachment = ActiveStorage::Attachment.find(params[:id])
          @record = @attachment.record
        end

        def authorize_attachment_parent!
          authorize! :manage, @record
        end
    end

Also be sure to wire this new controller up in your routes:

    # config/routes.rb
    scope :active_storage, module: :active_storage, as: :active_storage do
      resources :attachments, only: [:destroy]
    end

It’s also important to update the user interface to remove any reference to the attachment, and in this case I used Rails UJS. The javascript necessary to remove the element from my UI is simple and straightforward:

    // app/views/active_storage/attachments/destroy.js.erb
    document.getElementById("<%= dom_id @attachment %>").remove()

The only assumptions made here are that you had the representation of your attachment wrapped in a div with ID of attachment_#{id}, which you can easily render using Rails handy dom_id method.

You can now use the following markup next all of your attachments to allow for easy deletion:

    link_to "Delete", active_storage_attachment_path(attachment),
             method: :delete, remote: :true,
             data: { confirm: "Are you sure you wanna this?" }

Wrapping it Up

This approach is relatively simple, and relies heavily on the tools and features that Rails provides to us.

In my case I’m also leveraging a common partial to render thumbnails, metadata, and links for attachments, making it dead simple to add attachments to any model I choose to in the future.

As always things evolve over time, and while I’d like ot think I’m perfect I’m sure there are ways to improve this code, so if you have any suggestions or improvements I’d love to hear them.