-
-
Save ginjo/3688965 to your computer and use it in GitHub Desktop.
Recurring Job using Delayed::Job
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # # # # # scheduled_job.rb - recurring schedules for delayed_job.rb # # # # # | |
| # | |
| # This file is version controlled at https://gist.github.com/ginjo/3688965 | |
| # | |
| # Forked from https://gist.github.com/kares/1024726 | |
| # | |
| # This is an enhanced version of the original scheduled_job.rb | |
| # It was born out of the need to schedule a whole bunch of simple jobs. | |
| # I started with the sample below and quickly found that I was repeating | |
| # a lot of code. So I created the Delayed::Task pseudo-class that allows | |
| # simple creation of class-wrappers surrounding the code to be | |
| # scheduled. This gives you two easy ways to schedule any block of code. | |
| # | |
| # 1. Delayed::Task.schedule("MyTask", 30.minutes, Time.now){code_to_be_scheduled_starting_right_now} | |
| # => Wraps the code in a class MyTask, to be run every 30 minutes, starting now. | |
| # | |
| # 2. MyTask = Delayed::Task.new(6.hours){code_to_be_scheduled} | |
| # => Wraps the code in a class MyTask, to be run every 3.hours. | |
| # MyTask.schedule(5.minutes.from_now) | |
| # => start the schedule 5 minutes from now. | |
| # | |
| # MyTask.jobs # => show scheduled jobs | |
| # MyTask.unschedule # => stop scheduled jobs | |
| # | |
| # | |
| # I also added a couple of minor enhancements to the original ScheduledJob | |
| # module. | |
| # | |
| # | |
| # Instructions from the original Gist. | |
| # | |
| # Setup Your job the "plain-old" DJ (perform) way, include this module | |
| # and Your handler will re-schedule itself every time it succeeds. | |
| # | |
| # Sample : | |
| # | |
| # class MyTask | |
| # include Delayed::ScheduledJob | |
| # | |
| # run_every 1.day | |
| # | |
| # def display_name | |
| # "MyTask" | |
| # end | |
| # | |
| # def perform | |
| # # code to run ... | |
| # end | |
| # | |
| # end | |
| # | |
| # inspired by http://rifkifauzi.wordpress.com/2010/07/29/8/ | |
| # | |
| # Use: MyTask.schedule; MyTask.jobs | |
| # | |
| module Delayed | |
| module ScheduledJob | |
| def self.included(base) | |
| base.extend(ClassMethods) | |
| base.class_eval do | |
| @@logger = Delayed::Worker.logger | |
| cattr_reader :logger | |
| end | |
| end | |
| def perform_with_schedule | |
| perform_without_schedule | |
| schedule! # only schedule if job did not raise | |
| end | |
| # Schedule this "repeating" job | |
| def schedule!(run_at = nil) | |
| run_at ||= self.class.run_at | |
| if Gem.loaded_specs['delayed_job'].version.to_s.first.to_i < 3 | |
| Delayed::Job.enqueue self, 0, run_at | |
| else | |
| Delayed::Job.enqueue self, :priority=>0, :run_at=>run_at | |
| end | |
| end | |
| # Re-schedule this job instance | |
| def reschedule! | |
| schedule! Time.now | |
| end | |
| module ClassMethods | |
| def method_added(name) | |
| if name.to_sym == :perform && | |
| ! instance_methods(false).map(&:to_sym).include?(:perform_without_schedule) | |
| alias_method_chain :perform, :schedule | |
| end | |
| end | |
| def run_at | |
| run_interval.from_now | |
| end | |
| def run_interval | |
| @run_interval ||= 1.hour | |
| end | |
| def run_every(time) | |
| @run_interval = time | |
| end | |
| # | |
| # Show all jobs for this schedule | |
| def jobs | |
| if Rails::VERSION::MAJOR > 2 | |
| Delayed::Job.where("handler LIKE ?", "%#{name}%") | |
| else | |
| Delayed::Job.find(:all, :conditions=>["handler LIKE ?", "%#{name}%"]) | |
| end | |
| end | |
| # Remove all jobs for this schedule (Stop the schedule) | |
| def unschedule | |
| jobs.each{|j| j.destroy} | |
| end | |
| # Main interface to start this schedule (adds it to the jobs table). | |
| # Pass in a time to run the first job (nil runs the first job at run_interval from now). | |
| def schedule(run_at = nil) | |
| schedule!(run_at) unless scheduled? | |
| end | |
| def schedule!(run_at = nil) | |
| new.schedule!(run_at) | |
| end | |
| def scheduled? | |
| jobs.count > 0 | |
| end | |
| end # ClassMethods | |
| end # ScheduledJob | |
| # Task is a pseudo-class for creating named classes that represent any block of code to be scheduled. | |
| # | |
| # MyTask = Delayed::Task.new(5.minutes){do-something-here-every-5-minutes} | |
| # => creates a class MyTask that can be used to control the schedule of the encapsulated block. | |
| # | |
| # MyTask.schedule Time.now | |
| # => adds MyTask to the jobs table, and run the first job at Time.now. | |
| # | |
| # MyTask = Delayed::Task.new(2.hours){|*args_for_manual_run| puts args[0].to_s} | |
| # MyTask.run | |
| # => "" | |
| # | |
| # MyTask.run "something" | |
| # => "something" | |
| # | |
| module Task | |
| # Creates a new class wrapper around a block of code to be scheduled. | |
| def self.new(*args, &bloc) | |
| duration = args[0] || 1.day | |
| name = args[1] | |
| start_at = args[2] | |
| klas = Class.new | |
| klas.class_eval do | |
| include Delayed::ScheduledJob | |
| @in_duration = duration | |
| @in_bloc = bloc | |
| def self.in_duration; @in_duration; end | |
| def self.in_bloc; @in_bloc; end | |
| def self.run(*args); @in_bloc.call(*args); end | |
| def display_name | |
| self.class.name | |
| end | |
| def perform | |
| self.class.in_bloc.call | |
| end | |
| run_every duration | |
| end | |
| Object.const_set(name, klas) if name | |
| klas.schedule(start_at) if start_at and name | |
| return klas | |
| end | |
| # Schedule a block of code on-the-fly. | |
| # This is a friendly wrapper for using Task.new without an explicit constant assignment. | |
| # Delayed::Task.schedule('MyNewTask', 10.minutes, 1.minute.from_now){do_some_stuff_here} | |
| def self.schedule(*args, &bloc) | |
| self.new(args[1], args[0], args[2], &bloc) | |
| end | |
| end # Task | |
| end # Delayed | |
| # # # # # Control delayed_job workers with Upstart # # # # # | |
| # # | |
| # # Upstart config for Rails delayed_job worker. | |
| # # This gives a simple way to start/stop/restart daemons on Linux. | |
| # # This config defines an upstart control for a delayed_job worker (headless Rails instance). | |
| # # | |
| # # Place this config in a file in your project, | |
| # # then symlink this file in /etc/init/ as myprojectworker.conf. | |
| # # After installing new upstart script, run "initctl reload-configuration" | |
| # # (shouldn't need it, but seems to need it when using symlinks). | |
| # # Use: | |
| # # sudo start/stop/restart myprojectworker | |
| # | |
| # description "delayed_job worker for myproject" | |
| # author "[email protected]" | |
| # | |
| # start on (net-device-up | |
| # and local-filesystems | |
| # and started mysql | |
| # and runlevel [2345]) | |
| # stop on runlevel [016] | |
| # | |
| # respawn | |
| # | |
| # # Give up if restart occurs 10 times in 90 seconds. | |
| # respawn limit 10 90 | |
| # | |
| # #env RAILS_RELATIVE_URL_ROOT=/dev | |
| # #env RAILS_ENV=development | |
| # env RAILS_ENV=production | |
| # umask 007 | |
| # | |
| # # Default is 5 seconds | |
| # kill timeout 60 | |
| # | |
| # chdir /home/admin/sites/myproject | |
| # | |
| # exec su admin -c '/usr/bin/env script/delayed_job run' | |
| # # # # # Control delayed_job workers with Launchd # # # # # | |
| # | |
| # # Place this plist in ~/Library/LaunchDaemons/ as com.myproject.jobs.plist | |
| # # It should automatically start your delayed_job worker and keep it running. | |
| # # Manually start/stop with: | |
| # # launchctl <load|unload> -w ~/Library/LaunchDaemons/com.myproject.jobs.plist | |
| # | |
| # <?xml version="1.0" encoding="UTF-8"?> | |
| # <!DOCTYPE plist PUBLIC -//Apple Computer//DTD PLIST 1.0//EN | |
| # http://www.apple.com/DTDs/PropertyList-1.0.dtd > | |
| # <plist version="1.0"> | |
| # <dict> | |
| # <key>Label</key> | |
| # <string>com.myproject.jobs</string> | |
| # <key>WorkingDirectory</key> | |
| # <string>/Users/admin/sites/myproject</string> | |
| # <key>UserName</key> | |
| # <string>admin</string> | |
| # <key>ProgramArguments</key> | |
| # <array> | |
| # <string>/bin/bash</string> | |
| # <string>-c</string> | |
| # <string>script/delayed_job run</string> | |
| # </array> | |
| # <key>RunAtLoad</key> | |
| # <true/> | |
| # </dict> | |
| # </plist> | |
| # # # # # More ScheduledJob Examples # # # # # | |
| # | |
| # # You can load these lines along with (but after) the above modules. | |
| # # | |
| # # Schedule a proc. | |
| # task1 = proc{`Date >> log/mindless_task.log`} | |
| # Delayed::Task.schedule('MyTask1', 10.seconds, &task1) | |
| # | |
| # # Define a scheduled task and start the job, all in one line. | |
| # Delayed::Task.schedule('MyTask2', 10.seconds, Time.now){`Date >> log/mindless_task.log`} | |
| # | |
| # # Define a scheduled task that you can load into your Rails apps, without actually starting the schdules. | |
| # MyTask3 = Delayed::Task.new(10.seconds){` echo "MyTask3" >> log/mindless_task.log; Date >> log/mindless_task.log`} | |
| # MyTask4 = Delayed::Task.new(10.seconds){` echo "MyTask4" >> log/mindless_task.log; Date >> log/mindless_task.log`} | |
| # | |
| # # Only start the schedules when the worker daemon loads. | |
| # if $0[/delayed_job/i] | |
| # MyTask3.schedule | |
| # MyTask4.schedule(5.minutes.from_now) | |
| # end | |
Author
Thanks for the comment. Do you think there is a problem with this code that could cause a stack level too deep error, or was there something else going on with delayed_job?
Author
To follow up my original gist, I've been using this code for several weeks now to manage a dozen or so scheduled jobs. The whole mess has been very stable and appears to be superior to my old cron/rake solutions.
Also, I've had great results using Ubuntu's upstart init daemon to manage delayed_job workers. Really easy to setup too: Put an upstart conf file somewhere in your rails project and symlink to it from the /etc/init/ directory. Then you can do this at the linux prompt: sudo start/stop/restart myjobworker. I'll replace the helpers in this gist with an example of an upstart conf file. The equivalent tool on Mac is launchd, but I haven't used that for delayed_job workers yet.
@ginjo It was a configuration problem in development. I wanted to have the jobs execute right away and so I used
Delayed::Worker.delay_jobs = Rails.env.production?
however the processes were trying to schedule/run immediately and caused an infinite loop. It just took me a while to figure out why it was happening.
@ginjo Thanks for your work on this! I've borrowed your code, extended it a bit, and packaged it into a gem. Check it out if you're interested: https://github.com/amitree/delayed_job_recurring
@shawnpyle Among other things, the gem addresses the infinite loop issue.
Author
@afn Cool, I was just poking around to see the latest activity on delayed_job scheduling, thinking I might gem-ify this if no one else had yet. Thanks for picking it up (and fixing the loop issue), I'll check it out.
Also, just to comment on control of delayed_job workers and scheduled jobs, I've been using launchd on OSX and upstart on Ubuntu for a couple of years now, both with great success.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Thanks for posting this! Works great however I spent quite a bit of time trying to track down a "Stack too deep" error in my development environment. Come to find out, my delayed_job initializer contained the following to run delayed jobs right away:
Delayed::Worker.delay_jobs = Rails.env.production?Re-enabling delayed jobs for my development environment resolved that.