Quantcast
Channel: Korrelate Engineering Blog » Billy Watson
Viewing all articles
Browse latest Browse all 7

Restart Heroku Dynos on Your Terms

0
0

Heroku restarts all dynos at 24 hours of uptime. This is well-known in the community and is hopefully not the first time you’re finding this out. Restarting a web dyno may have small effects to your user base, but in our case has serious ramifications if it happens on a worker dyno during business hours. A few of our background jobs have the potential to take an hour or more, and some of those jobs are kicked off by users. In an ideal world, we wouldn’t have these long-running tasks; unfortunately, we live in the real world, so we have to take action.

The goal is to control when Heroku restarts the dynos so that there are no surprises. First, we have to be able to restart them, period. To do that, we’ll need the heroku-api Gem, a few config variables, a rake task, and a plugin. Sound complicated? Not really…

The Code

First, the Gemfile:

gem 'heroku-api'

Next we need a task to use this, so in the Rakefile:

namespace :heroku do
  desc 'restarts all the heroku dynos so we can control when they restart'
  task :implode do
    Heroku::API.
      new(username: ENV['HEROKU_USERNAME'], password: ENV['HEROKU_PASSWORD']).
      post_ps_restart(ENV['HEROKU_APP_NAME'])
  end
end

Alternatively, you can use an API token so your coworkers won’t know the username and password of your main account. Then, we’ll need to set up those config variables so the task doesn’t implode, itself:

heroku config:set HEROKU_USERNAME=[username] HEROKU_PASSWORD=[password] HEROKU_APP_NAME=[app_name] -a [app_name]

Now, go ahead and deploy and test:

git push [heroku_remote_name] [feature_branch]:master
heroku run rake heroku:implode -a [app_name]

Lastly, we’ll need to set up the task to run this on schedule. I’ve chosen to go with the free Heroku cron add-on:

heroku addons:add scheduler:standard -a [app_name]
heroku addons:open scheduler -a [app_name]

This will open up the scheduler UI in your browser and you can create a scheduled worker to run the rake task whenever you’d like. We only need it once per day and we’re choosing to run it before our first scheduled job of the day.

schedule_implosion

Now you can enjoy your newfound control and comfort.

 

Sources

https://github.com/heroku/heroku.rb

https://devcenter.heroku.com/articles/scheduler#scheduling-jobs

https://devcenter.heroku.com/articles/authentication#api-token-storage


Viewing all articles
Browse latest Browse all 7

Latest Images

Trending Articles





Latest Images