You are currently browsing the category archive for the ‘Gems’ category.
Just finished presenting Tap at this year’s RubyConf. Here are the slides.
Tap –[Not] a Talk About Replacing Rake
I ran into trouble when I tried turning rake into something it is not. Rake is a build program designed for build-like tasks; rake is not a platform for general-purpose task libraries. Given it’s design goals, rake very sensibly does not facilitate extensive documentation (who needs it to compile something), inputs (although this has changed somewhat), configuration, testing, or distribution.
It’s also a dependency-based system; workflows constructed by rake are synthesized in reverse — it’ll be you, not your program, that gets forked when you try to make an imperative workflow. It’s simply the nature of rake! Rake is an excellent build program, but these types of things are in a different domain.
Tap (Task Application)
Tap was originally designed as a simple workflow engine, but it’s evolved into a general-purpose framework for creating configurable, distributable task libraries. Tap tasks can be defined in much the same way as a Rake task:
# Goodnight::manifest your basic goodnight moon task # Prints the input with a configurable message. Tap.task 'goodnight', {:message=> 'goodnight'} do |task, input| task.log task.message, input "#{task.message} #{input}" end
Tap pulls documentation out of task declarations to generate manifests:
% tap run -T sample: goodnight # your basic goodnight moon task tap: dump # the default dump task rake # run rake tasks
And help:
% tap run -- goodnight --help Goodnight -- your basic goodnight moon task ------------------------------------------------------------------ Says goodnight with a configurable message. ------------------------------------------------------------------ usage: tap run -- goodnight NAME configurations: --message MESSAGE a goodnight message options: -h, --help Print this help --name NAME Specify a name --use FILE Loads inputs from file
Tasks are immediately available to run with inputs and configurations:
% tap run -- goodnight moon I[00:09:55] goodnight moon % tap run -- goodnight moon --message hello I[00:10:01] hello moon
Task declarations define classes which naturally support namespaces, subclassing and testing. When the shorthand declaration is not enough, task classes can be defined in the standard way:
# Hello::manifest a hello world task # A more complicated hello world task illustrating # config blocks and a full task class definition. # class Hello < Tap::Task config :greeting, 'hello', &c.string # a greeting string config :reverse, false, &c.flag # maps to a flag def process(name) message = reverse ? greeting.reverse : greeting log message, name "#{message} #{name} result" end end task = Hello.new task.process('world') # => "hello world result" task.reverse = true task.process('world') # => "olleh world result" task.greeting = :symbol # !> ValidationError
Configurations map to methods and can utilize a validation/transformation block. Tap defines a number of common blocks (ex c.integer, c.regexp, etc.) that may also imply metadata for the command line (ex c.flag):
% tap run -- hello world --reverse I[20:04:33] olleh world
Distribution
Tap supports distribution of tasks as gems. To illustrate, say we installed the sample_tasks gem. Now our manifest looks like this:
% tap run -T sample: goodnight # your basic goodnight moon task hello # a hello world task sample_tasks: concat # concatenate files with formatting copy # copies files grep # search for lines matching a pattern print_tree # print a directory tree tap: dump # the default dump task rake # run rake tasks
Tap checks the installed gems for a ‘tap.yml’ configuration file or a ‘tapfile.rb’ task file; any gems with one (or both) of these files gets pulled into the execution environment. Now tasks can be specified either by a short name when there isn’t a name conflict (ex goodnight, print_tree), or by a full name that includes the environment (ex sample:goodnight, sample_tasks:print_tree).
% tap run -- sample_tasks:print_tree . . |- Rakefile |- lib |- sample.gemspec |- tapfile.rb `- test |- tap_test_helper.rb |- tap_test_suite.rb `- tapfile_test.rb
Not bad, eh?
Workflows and the Roadmap
Tap support simple workflows in the imperative style. Tasks can be assigned an on_complete block that executes when the task completes, allowing results to be examined and new tasks to be enqued as needed.
app = Tap::App.instance t1 = Tap.task('t1') {|t| 'hellO'} t2 = Tap.task('t2') {|t, input| input + ' woRld' } t3 = Tap.task('t3') {|t, input| input.downcase } t4 = Tap.task('t4') {|t, input| input.upcase } t5 = Tap.task('t5') {|t, input| input + "!" } # sequence t1, t2 app.sequence(t1, t2) # fork t2 results to t3 and t4 app.fork(t2, t3, t4) # unsynchronized merge of t3 and t4 into t5 app.merge(t5, t3, t4) app.enq(t1) app.run app.results(t5) # => ["hello world!", "HELLO WORLD!"]
True support for workflows from the command line is lacking right now, but it will be coming soon. Here’s a short list of what else is planned:
- Global Rake Tasks (hopefully!). I should be able to find and load rakefiles using the Tap execution environment. Tap already allows you to incorporate local rake tasks into a workflow using the ‘rake’ task.
- Tap server. In a small-group environment where some people are computer savvy and others aren’t, it would be really useful to serve up your tasks using a web interface.
- More test support. Tap provides several modules for testing tasks and supporting common types of tasks (ex file transformation tasks). Currently the modules are a bit incomplete, and they’re only geared towards Test::Unit. I’d like to add support for RSpec in the future.
This last summer I finished the joinfix gem providing a solution to the fixture join problem — mainly that it’s a pain to create fixtures by specifying entry ids across multiple fixture files. When Mike Clark and Chad Fowler opened up submissions for Advanced Rails Recipies, I sent JoinFix to them and it was accepted.
I took a look at the new Foxy Fixtures in Rails 2.0 and was shocked/pleased to find something similar has gotten incorporated directly into the core. Goes to show how problematic fixtures really were. JoinFix still is interesting, however, mainly because JoinFix lets you define entries inline.
Consider the following data model:
class User < ActiveRecord::Base has_many :user_groups has_many :groups, :through => :user_groups end class Group < ActiveRecord::Base has_many :user_groups has_many :users, :through => :user_groups end class UserGroup < ActiveRecord::Base belongs_to :user belongs_to :group end
You can write your fixtures using the naming scheme you lay out in your models, referencing entries across multiple fixture files (similar to Foxy) or you can define them inline:
[users.yml] john_doe: full_name: John Doe groups: admin_group # => reference to the 'admin_group' entry jane_doe: full_name: Jane Doe groups: # => you can specify an array of entries if needed - admin_group - worker_group: # => inline definition of the 'worker_group' entry name: Workers [groups.yml] admin_group: # => the referenced 'admin_group' entry id: 3 # => you can (but don't have to) specify ids name: Administrators
Join entries implied in your definition, as in a has_and_belongs_to_many association, will be created and named by joining together the names of the parent and child, ordered by the ’<’ operator. For example, the users.yml and groups.yml fixtures produce these entries:
[users] john_doe: id: 1 # => primary keys are assigned to all entries full_name: John Doe jane_doe: id: 2 full_name: Jane Doe [groups] admin_group: id: 3 name: Administrators worker_group: id: 1 name: Workers [user_groups] admin_group_john_doe id: 1 user_id: 1 # => references are resolved to their foreign keys group_id: 3 # => explicitly set primary keys are respected admin_group_jane_doe id: 2 user_id: 2 group_id: 3 jane_doe_worker_group # => Notice the '<' operator in action id: 3 user_id: 2 group_id: 1
Nesting is allowed. This will make the same entries as above:
[users.yml] john_doe: full_name: John Doe groups: admin_group: id: 3 name: Administrators users: jane_doe: full_name: Jane Doe groups: worker_group: name: Workers
In this final form, JoinFix defines a highly-involved fixture in one chunk, in one file. This can be a BIG advantage when you try to test some cross-table, complicated lookup. The full fixture is centralized and easy to manage.