Adding support for cross-cluster associations to Rails 7

At GitHub, we recently added a new feature to Rails that will be available in 7.0: support for handling associations across database clusters. Read about it on the blog.

Upgrading GitHub to Ruby 2.7

Read about GitHub’s Ruby 2.7 upgrade at https://github.blog/2020-08-25-upgrading-github-to-ruby-2-7/.

Running GitHub on Rails 6.0

Read about GitHub’s Rails 6.0 upgrade at https://github.blog/2019-09-09-running-github-on-rails-6-0/.

Upgrading GitHub from 3.2 to 5.2

Read about GitHub’s Rails upgrade at https://github.blog/2018-09-28-upgrading-github-from-rails-3-2-to-5-2/.

HTTP2 Early Hints

Update: The PR’s discussed in this blog post have been merged. You can try out Early Hints by using Rails and Puma master. You’ll need to also use Arel master until Rails 5.2 is released.

Recently I’ve been working with Aaron Patterson on implementing the HTTP2 Early Hints.

The spec for Early Hints is still in draft but explains the requirements for implementation. In order to start preparing Rails and Rack for HTTP2 we’ve implemented Early Hints handling in Puma and Rails.

As of today (September 29, 2017) the pull requests haven’t been merged but we’d love for you to try them out if you’re interested in implementing Early Hints. A lot of this work is somewhat experimental so try it on production at your own risk.

In this post I’ll go over what early hints are, what the spec expects, and finally how to use the two pull requests to test it out on your own application.

Early Hints, what huh?

Early Hints is a new HTTP status code that allows your application to send links to assets that you would like to load early. For example if you have a stylesheet and javascript file included in your Rails application you may want to tell the client about them early so it can start downloading them.

The Early Hints spec contains header Links with references to those resources that you need downloaded early. An example response from the server looks like this:

 HTTP/1.1 103 Early Hints
 Link: </style.css>; rel=preload; as=style
 Link: </script.js>; rel=preload; as=script

 HTTP/1.1 200 OK
 Date: Fri, 26 May 2017 10:02:11 GMT
 Content-Length: 1234
 Content-Type: text/html; charset=utf-8
 Link: </style.css>; rel=preload; as=style
 Link: </script.js>; rel=preload; as=script

There are 2 responses, one for Early Hints that sends the linked resources and one for the final request that should also include the same links.

For the Link header, Link: </style.css>; rel=preload; as=style the style.css is the path to the css file, rel does not change and should be preload, and as is the type of resource you’re sending. The as is not required, but is useful for content-security policies.

While HTTP/1.1 can accept a 103, a server might not actually send the hints unless it absolutely knows they will be accepted. HTTP/2 is less likely to have this problem as handling of response headers doesn’t affect how the response body is determined (as noted in the spec.

Implementing Early Hints with Puma and Rails

The Puma PR and Rails PR are both a work in progress so they are likely to change before we merge them into their respective projects.

Below I’ll describe how you can try these two out in your Rails app. First make sure you’re using the Rails and Puma gems set to the branch for these PRs’

# Gemfile

gem 'rails', git: 'https://github.com/eileencodes/rails', branch: 'early-hints'
gem 'puma', git: 'https://github.com/eileencodes/puma', branch: 'early-hints'
gem 'arel', git: 'https://github.com/rails/arel'

Next you’ll need a proxy that can handle HTTP/2.

Install h2o via homebrew or the method you need for your computer. I used homebrew with brew install h2o.

Find the h2o config file (mine is in /usr/local/etc/h2o/h2o.conf and update it with the following:

hosts:
  "localhost":
    listen:
      port: 9090
      ssl:
        certificate-file: /usr/local/etc/h2o/localhost.crt
        key-file: /usr/local/etc/h2o/localhost.key
    paths:
      "/":
        proxy.reverse.url: "http://127.0.0.1:3000/"

access-log: /usr/local/var/h2o/access-log
error-log: /usr/local/var/h2o/error-log

Next you’ll need a “fake” certificate so you can do HTTPS requests with localhost. Otherwise Early Hints won’t work. See this post for more information on the following.

Run this command:

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout localhost.key -out localhost.crt

And follow the prompts. I hit enter for everything. Then move your certificate and key into your h2o folder since that’s where the config is looking. If you want the cert to live somewhere else, update your h2o.conf with that new path.

Now start the h2o server with the config: h2o -c h2o/h2o.conf

And start your Rails server (if it’s a Rails 5 app you will automatically be using Puma. This will not work if you’re not using the Puma branch I linked above): bin/rails s --early-hints

Open Chrome and go to https://localhost:9090/someurl replacing someurl with a URL you know uses a javascript_include_tag or a stylesheet_link_tag because that’s the way Early Hints is currently implemented in Rails.

Open the Chrome dev tools and go to the network tab. From there you should see Push / posts where posts is the URL you entered under the “Initiator” column. Below is a screenshot of my Chrome dev tools showing the Early Hints loading. If you click the image to load the full side it is easier to view.

My Process for Writing a Talk

There’s a ton of blog posts out there with advice on how to write talks. They all discuss the “right way”, but in reality I don’t think anyone follows a process prescribed by someone else.

I’m a very visual person and in a way I write talks backwards. “The right way” would be to write copy and then build the slides around it. I shouldn’t even be opening Keynote until I have solid, well-articulated copy. But that’s not how I’ve been writing talks.

I’m definitely not recommending you follow my way of doing things, but I thought it would be fun to share my process. I’ve developed approximately 6 talks between meetups and conferences. I put a lot of time and effort into each one. My How to Performance talk was probably 100-120+ hours depending on if you count revisions and practice I did before each conference.

Here’s my process:

1. Talk to anyone who will listen about something I’m passionate about or working on recently. The talk I’m currently writing is about Security. I’ve ranted and explained all sort of things I believe about security and patching applications to anyone who will listen; my dog, my husband, my friends and coworkers. If I keep talking about it enough, I know I care about it enough to turn it into a talk.

2. Collect ideas in a Google doc so I can access the file anywhere.

3. Come up with a title (see I told you this was backwards because this is “supposed to” be after an abstract)

4. Write an abstract.

   • Write something, anything down that resembles a beginning, middle and end of an abstract

   • Show it to my husband who tells me how to write a better one

   • State that I can’t do this, I don’t know how to do this

   • Go to the gym and work out my frustration

   • Come back and bang out a kick-ass abstract

5. Write an outline. This is where it starts to get weird.

6. Open Keynote and pick colors. Colors are very important to me. I was a photography major and have a design and art background. If the colors don’t feel right I can’t write the talk.

7. Collect memes and gifs to help express myself.

8. Build out the middle slides; the meat. This is generally where I start adding all the gifs I’ve saved over the past couple months that I think would go great with sentiments I have in the talk.

9. Freak out that this talk isn’t going well and it will never be good. I’ve given 3 well liked talks at 8 conferences and I still believe they were a one-hit wonder and no one will like any future talks. This is something I deal with every time, and it’s hard. It’s difficult to tell yourself you’re going to be fine. That’s why it’s good to have a great support system of family and friends who will help pull your head out of your ass so you can keep working.

10. Start writing copy for the beginning slides I never added. Add slides to fill in the opening of the talk as I write them.

11. Write the copy for the ending slides I never added. Add slides to fill in the end of the talk as I write them.

12. Go back to the middle slides and write the copy for those adding, removing, and reordering slides as necessary. Once I’m satisfied the talk is “written” I go back and edit copy. Written to me just means I could go give this talk as is and it would be complete, but confusing. It’s not perfect but all my main points are in there.

13. Give talk at a local meetup.

14. Make changes based on the meetup feedback.

15. Practice. Edit. Practice. Every night for 2-3 weeks leading up to the conference.

16. Give talk at a conference.

17. Make changes based on things I felt were confusing and based on questions/feedback I get.

18. Repeat 15-17 until I retire the talk.

This process, while convoluted at points, is my process. This works for me. Don’t let anyone tell you you’re doing it wrong. It’s your talk. If you a) get up and actually do it or b) people find your talk interesting, then you were successful. Find what works for you and what gets you up there on that stage. That’s all the matters.

I’m giving talks at 4 conferences so far this year. Mountain West Ruby Conference in March, Twilio’s SignalConf in May, Brighton Ruby Conference in July, and AbstractionsConf in August. You can always find what conferences I’ll be at on my speaking page.

Rails 5: The Sprockets 4 Manifest

When Rails 5 is released it will require that you upgrade to Sprockets 4. Sprockets 4 has some big changes in how it knows what assets to compile so you will definitely have some updates to make if you rely on Sprockets within Rails. Since there are a lot of changes in Sprockets 4 I’ll just be talking about the new manifest.js in this post.

With Sprockets you used to tell your config/initializers/assets.rb what assets were supposed to be precompiled. In Sprockets 4 you will use a manifest.js inside your app/assets/ directory to tell Sprockets what assets to precompile. This behavior is actually available to use in Sprockets 3 but sprockets-rails has a conditional that only activates this in Sprockets 4.

# sprockets-rails/lib/sprockets/railtie.rb

if using_sprockets4?
  config.assets.precompile  = %w( manifest.js )
else
  config.assets.precompile  = [LOOSE_APP_ASSETS, /(?:\/|\\|\A)application\.(css|js)$/]
end

Note: the file type that manifest.js is stored as may change. There is disucssion around moving the manifest.js to manifest.yml because it doesn’t make sense that the manifest has a file type that is unrelated to it’s usage.

So, how do you use a the new manifest file? In your app/assets/ directory add a new directory named config/. Inside that folder add a file called manifest.js.

In the manifest.js you’ll want to link your JS and CSS directories as well as any other directories you rely on like images, fonts, sounds etc.

Here is an example of a manifest.js that links JS, CSS, fonts, and images.

// JS and CSS bundles
//
//= link_directory ../javascripts .js
//= link_directory ../stylesheets .css


// Images and fonts so that views can link to them
//
//= link_tree ../fonts
//= link_tree ../images

Previously you didn’t need to include fonts and images in your precompiled assets list and Sprockets Rails would use the LOOSE_APP_ASSETS constant to figure out those items, but now you have to explicitly include them in your config.

You’ll notice that we use link_directory for CSS and JS, then the path from the config manifest file to the javascript “file”. We compile .coffee and .scss down to .js and .css, respectively, so we denote that after the path to the JS and CSS files to tell Sprockets what to compile them into. Images and fonts don’t change file type when compiled.

Once you’ve done that you’ll need to remove config.assets.precompile from your config/initializers/assets.rb.

config.assets.precompile += %w( 
  all.css all.js
)

Smaller apps may only have the precompile directive in their applications so in that case you can delete the config/initializers/assets.rb file. For larger apps, like Basecamp we have some extra settings regarding assets and didn’t want to delete the config file.

Rails 5: ActionController::Parameters Now Returns an Object Instead of a Hash

A big change coming is how ActionController::Parameters works. ActionController::Parameters is where all of your params come from for your controllers. Calling params used to return a hash, but now will return an object.

Note: this doesn’t affect accessing the keys in the params hash like params[:id]. You can view the PR that implemented this change here: https://github.com/rails/rails/pull/20868

To access the parameters in the object you can add #to_h to the parameters:

params.to_h

If those params aren’t explictly permitted you will be returned a hash with only the permitted parameters. If none are permitted you’ll get an empy hash ({}). This comes in where you may be running #symbolize_keys or #slice on unpermitted params. If you’re accessing params that aren’t being saved to a model/db then you probably aren’t explictly permitting those parameters.

If we look at the #to_h method in ActionController::Parameters we can see it checks if the parameters are permitted before converting them to a hash.

# actionpack/lib/action_controller/metal/strong_parameters.rb
def to_h
  if permitted?
    @parameters.to_h
  else
    slice(*self.class.always_permitted_parameters).permit!.to_h
  end
end

Let’s take an example where we are slicing params to use later. If we have this method that slices params we used to be able to write:

def do_something_with_params
  params.slice(:param_1, :param_2)
end

Which would return:

  { :param_1 => "a", :param_2 => "2" }

But now that will return an ActionController::Parameters object.

Calling #to_h on this would return an empty hash because param_1 and param_2 aren’t permitted.

To get access to the params from ActionController::Parameters you need to first permit the params and then call #to_h on the object. The following returns the same thing as slice did previously.

def do_something_with_params
  params.permit([:param_1, :param_2]).to_h
end

Another way to do this would be to use #to_unsafe_hash if you know the params are not user supplied and are safe:

def do_something_with_params
  params.to_unsafe_h.slice(:param_1, :param_2)
end

By default controller and action parameters are allowed. To explicitly always allow other parameters you can set a configuration option in your application.rb that allows those parameters. Note: this will return the hash with string keys, not symbol keys.

Config option:

  config.always_permitted_parameters = %w( controller action param_1 param_2 )

Calling slice on the parameters:

def do_something_with_params
  params.slice("param_1", "param_2").to_h
end

If you’re not sure when you’ll have time to upgrade it would be a good idea to write some tests for your controllers that access the params. That way when you do upgrade you’ll know to fix the params because your tests will be failing.

Website Redesign with Jekyll and Github Pages

Back in 2012 I built my first Rails app from scratch. It was a little blog that was meant as a learning tool and a way to contribute back to a community that gives so much to beginngers (gems, mentoring, etc).

I learned a lot from that blog, it helped me get jobs, and kept me accountable for the things I learned. It served as both a place to try new features and to record things I learned. It was where I built my first admin system, wrote about learning to build a forgot password form, everything.

But I realized I was neglecting it. I dreaded Rails security vulnerabilities, server vulnerabilities, and the chore that upgrading it to the next verison of Rails was going to be. It was also in desperate need of a design overhaul and didn’t look great on mobile browsers.

A few weeks ago I decided that I was tired of all of the above, and that my little blog had served it’s purpose well more than I had expected too. It helped me grow my career and myself.

Earlier today I swapped my blog to Github pages. I have to say I am very impressed with Jekyll even though I’ll miss my little Rails blog that I built from scratch. Jekyll is very powerful with great documentation. I was able to basically recreate my blog with the same URLs, RSS feed, and all. I have a front-end background so I was able to make my own theme quickly and easily.

The one caveat is that Github pages runs on a different version of Jekyll so some of the things in the documentation aren’t correct for that version. Once I figure that out though it was smooth sailing.

For posterity, here’s a picture of the old blog. I’m going to miss it.

Getting Your Local Environment Setup to Contribute to Rails

At RailsConf I’ll be leading a workshop on contributing to Ruby on Rails called “Breaking Down the Barrier: Demystifying Contributing to Rails”. My goal is to help you be confident in your ability to contribute to Rails. I’ll be focusing on contributing guidelines, advanced git commands, and traversing unfamiliar source code. I’ve allotted 90 minutes for the workshop so in order for you to get the most out of it you should have your system ready. In this post I’m going to go over the basics of getting set up.

Technically, the easiest way to get Rails running locally is to use the supplied VM. I prefer to have it running on my local machine, but if you’re using Windows I highly recommend using the VM. Although the VM is referred to the “easy way”, you likely already have 50% of the things you need installed on your system already if you’re actively developing Rails applications. If you do decide to use the VM you can skip these instructions. Please contact me on eileencodes to let me know if the VM directions are wrong. Or better yet, if you figure it out send a pull request!

Let’s get started with the basics of getting set up!

Ruby Manager

When working with Rails it’s likely you’ll be using a different Ruby version than you use in your production applications. It’s best to use Rails master with the most up-to-date version of Ruby. Currently, you can’t use Rails master / future Rails 5 without Ruby 2.2.2. You probably already have a way to have multiple rubies installed on your machine with either rbenv, rvm or chruby.

I personally use rbenv but I’ve used rvm in the past and hear good things about chruby. It really doesn’t matter which one you use as long as you can have multiple rubies installed on your machine.

Once you have that set up, install Ruby 2.2.1 2.2.2 (2.2.1 had a security vulnerability). Don’t forget if you have a new version of Ruby you’ll need to install bundler before you run bundle install on the Rails repo.

Fork & Clone Rails

Now we’ll get the Rails source code set up. First go to [github.com/rails/rails][rails-repo]{:target=”_blank”} and click “fork”. Some developers prefer to clone the main Rails repo and set up their fork as an upstream, but unless you have push rights to Rails (a commit bit) I don’t think this really makes sense. In my opinion having origin set as your repo works best so this guide is going to show you my preferred method.

Checkout your version of Rails to your local machine:

$ git clone git@github.com/your-fork/rails.git

You’ll need to get Rails master main repo as an upstream to your Rails. To do this simply run:

$ git remote add upstream https://github.com/rails/rails.git

Anytime you want to pull changes from Rails master into your master do:

$ git pull --rebase upstream master
$ git push origin master

Here you are pushing to your origin so your remote origin is always up-to-date with your master branch. Don’t work on your master branch and send PR’s from there. Always create a new branch. That way you can be working on multiple patches and your master is always clean and ready to checkout a new branch from. Pushing to your origin master also makes it easy to reset any of your branches to master without having to re-pull changes from upstream Rails.

Don’t forget to add a .ruby-version file to your Rails repo, but be sure not to check this in. I have a .gitignore_global file that sits in my home directory and ignores all .ruby-version files. Then you should run bundle install.

Databases

Ok now that you’ve got the Ruby and Rails source, you’ll need to get a few more things installed before you can start running Rails tests. And the most important of those things is databases!

It’s not really a requirement that you have ALL the databases installed but it’s a good idea to have the default databases that the Active Record supports; SQLite3, MySQL, and PostgreSQL. This will help you test the main adapters that are supported in Rails. It’s also a good idea if you’re working on any SQL specific parts of Active Record; you want to be sure you aren’t negatively changing the behavior of those other databases.

How you install databases is up to you. As a OS X user I install them with homebrew and follow the instructions output after installation. Remembering all the start/stop commands for databases is a pain though so I use LaunchRocket to control this. It’s a OS X preference pane to manage databases installed with homebrew. Additionally, you’ll need memcached for some ActionDispatch and ActionController tests.

Once you have the necessary databases installed you’ll need to create the databases and users required by the Rails tests.

MySQL

First create the users

$ mysql -uroot -p

mysql> CREATE USER 'rails'@'localhost';
mysql> GRANT ALL PRIVILEGES ON activerecord_unittest.*
       to 'rails'@'localhost';
mysql> GRANT ALL PRIVILEGES ON activerecord_unittest2.*
       to 'rails'@'localhost';
mysql> GRANT ALL PRIVILEGES ON inexistent_activerecord_unittest.*
       to 'rails'@'localhost';

Then create the databases

$ cd activerecord
$ bundle exec rake db:mysql:build

PostgreSQL

If you’re a Linux user create the user by running:

$ sudo -u postgres createuser --superuser $USER

If you’re an OS X user run:

$ createuser --superuser $USER

And then create the databases:

$ cd activerecord
$ bundle exec rake db:postgresql:build

Creating and Destroying

It’s also possible to create both MySQL and PostgresSQL databases at the same time by running:

$ cd activerecord
$ bundle exec rake db:create

And you can destroy the databases and start over with:

$ cd activerecord
$ bundle exec rake db:drop

Running the Tests

Now that you have Ruby, the Rails source code, and the databases installed it’s time to run the tests. Now don’t just run rake test in the Rails root directory because you will be there all day waiting for railties tests to finish. Simply cd into the library you want to test and run:

$ rake test

To run Active Record tests, be sure to include the database adapter you want to test or else sqlite3, mysql, mysql2, and postgresql adapter tests will all run. To run tests for specific adapters do the following:

$ bundle exec rake test:sqlite3
$ bundle exec rake test:mysql
$ bundle exec rake test:mysql2
$ bundle exec rake test:postgresql

And don’t forget all these commands are available if you run rake -T in the directory you’re in.

See You at the Workshop

I’d tell you more about contributing to Rails but then I would have nothing to talk about at the workshop! I know it will be a lot of fun and you’ll learn tons. To read more about my workshop go to the RailsConf website.

If you have any issues at all getting set up ping me on twitter at eileencodes and I’ll do my best to point you in the right direction.