You are currently browsing the category archive for the ‘Gems’ category.
Just finished presenting Tap at this year’s RubyConf. Here are the slides.
Tap –[Not] a Talk About Replacing Rake
Advertisements
I ran into trouble when I tried turning rake into something it is not. Rake is a build program designed for build-like tasks; rake is not a platform for general-purpose task libraries. Given it’s design goals, rake very sensibly does not facilitate extensive documentation (who needs it to compile something), inputs (although this has changed somewhat), configuration, testing, or distribution.
It’s also a dependency-based system; workflows constructed by rake are synthesized in reverse — it’ll be you, not your program, that gets forked when you try to make an imperative workflow. It’s simply the nature of rake! Rake is an excellent build program, but these types of things are in a different domain.
Tap (Task Application)
Tap was originally designed as a simple workflow engine, but it’s evolved into a general-purpose framework for creating configurable, distributable task libraries. Tap tasks can be defined in much the same way as a Rake task:
# Goodnight::manifest your basic goodnight moon task
# Prints the input with a configurable message.
Tap.task 'goodnight', {:message=> 'goodnight'} do |task, input|
task.log task.message, input
"#{task.message} #{input}"
end
Tap pulls documentation out of task declarations to generate manifests:
% tap run -T
sample:
goodnight # your basic goodnight moon task
tap:
dump # the default dump task
rake # run rake tasks
And help:
% tap run -- goodnight --help
Goodnight -- your basic goodnight moon task
------------------------------------------------------------------
Says goodnight with a configurable message.
------------------------------------------------------------------
usage: tap run -- goodnight NAME
configurations:
--message MESSAGE a goodnight message
options:
-h, --help Print this help
--name NAME Specify a name
--use FILE Loads inputs from file
Tasks are immediately available to run with inputs and configurations:
% tap run -- goodnight moon
I[00:09:55] goodnight moon
% tap run -- goodnight moon --message hello
I[00:10:01] hello moon
Task declarations define classes which naturally support namespaces, subclassing and testing. When the shorthand declaration is not enough, task classes can be defined in the standard way:
# Hello::manifest a hello world task
# A more complicated hello world task illustrating
# config blocks and a full task class definition.
#
class Hello < Tap::Task
config :greeting, 'hello', &c.string # a greeting string
config :reverse, false, &c.flag # maps to a flag
def process(name)
message = reverse ? greeting.reverse : greeting
log message, name
"#{message} #{name} result"
end
end
task = Hello.new
task.process('world') # => "hello world result"
task.reverse = true
task.process('world') # => "olleh world result"
task.greeting = :symbol # !> ValidationError
Configurations map to methods and can utilize a validation/transformation block. Tap defines a number of common blocks (ex c.integer, c.regexp, etc.) that may also imply metadata for the command line (ex c.flag):
% tap run -- hello world --reverse I[20:04:33] olleh world
Distribution
Tap supports distribution of tasks as gems. To illustrate, say we installed the sample_tasks gem. Now our manifest looks like this:
% tap run -T
sample:
goodnight # your basic goodnight moon task
hello # a hello world task
sample_tasks:
concat # concatenate files with formatting
copy # copies files
grep # search for lines matching a pattern
print_tree # print a directory tree
tap:
dump # the default dump task
rake # run rake tasks
Tap checks the installed gems for a ‘tap.yml’ configuration file or a ‘tapfile.rb’ task file; any gems with one (or both) of these files gets pulled into the execution environment. Now tasks can be specified either by a short name when there isn’t a name conflict (ex goodnight, print_tree), or by a full name that includes the environment (ex sample:goodnight, sample_tasks:print_tree).
% tap run -- sample_tasks:print_tree .
.
|- Rakefile
|- lib
|- sample.gemspec
|- tapfile.rb
`- test
|- tap_test_helper.rb
|- tap_test_suite.rb
`- tapfile_test.rb
Not bad, eh?
Workflows and the Roadmap
Tap support simple workflows in the imperative style. Tasks can be assigned an on_complete block that executes when the task completes, allowing results to be examined and new tasks to be enqued as needed.
app = Tap::App.instance
t1 = Tap.task('t1') {|t| 'hellO'}
t2 = Tap.task('t2') {|t, input| input + ' woRld' }
t3 = Tap.task('t3') {|t, input| input.downcase }
t4 = Tap.task('t4') {|t, input| input.upcase }
t5 = Tap.task('t5') {|t, input| input + "!" }
# sequence t1, t2
app.sequence(t1, t2)
# fork t2 results to t3 and t4
app.fork(t2, t3, t4)
# unsynchronized merge of t3 and t4 into t5
app.merge(t5, t3, t4)
app.enq(t1)
app.run
app.results(t5) # => ["hello world!", "HELLO WORLD!"]
True support for workflows from the command line is lacking right now, but it will be coming soon. Here’s a short list of what else is planned:
- Global Rake Tasks (hopefully!). I should be able to find and load rakefiles using the Tap execution environment. Tap already allows you to incorporate local rake tasks into a workflow using the ‘rake’ task.
- Tap server. In a small-group environment where some people are computer savvy and others aren’t, it would be really useful to serve up your tasks using a web interface.
- More test support. Tap provides several modules for testing tasks and supporting common types of tasks (ex file transformation tasks). Currently the modules are a bit incomplete, and they’re only geared towards Test::Unit. I’d like to add support for RSpec in the future.
This last summer I finished the joinfix gem providing a solution to the fixture join problem — mainly that it’s a pain to create fixtures by specifying entry ids across multiple fixture files. When Mike Clark and Chad Fowler opened up submissions for Advanced Rails Recipies, I sent JoinFix to them and it was accepted.
I took a look at the new Foxy Fixtures in Rails 2.0 and was shocked/pleased to find something similar has gotten incorporated directly into the core. Goes to show how problematic fixtures really were. JoinFix still is interesting, however, mainly because JoinFix lets you define entries inline.
Consider the following data model:
class User < ActiveRecord::Base has_many :user_groups
has_many :groups, :through => :user_groups
end
class Group < ActiveRecord::Base
has_many :user_groups
has_many :users, :through => :user_groups
end
class UserGroup < ActiveRecord::Base
belongs_to :user
belongs_to :group
end
You can write your fixtures using the naming scheme you lay out in your models, referencing entries across multiple fixture files (similar to Foxy) or you can define them inline:
[users.yml]
john_doe:
full_name: John Doe
groups: admin_group # => reference to the 'admin_group' entry
jane_doe:
full_name: Jane Doe
groups: # => you can specify an array of entries if needed
- admin_group
- worker_group: # => inline definition of the 'worker_group' entry
name: Workers
[groups.yml]
admin_group: # => the referenced 'admin_group' entry
id: 3 # => you can (but don't have to) specify ids
name: Administrators
Join entries implied in your definition, as in a has_and_belongs_to_many association, will be created and named by joining together the names of the parent and child, ordered by the ’<’ operator. For example, the users.yml and groups.yml fixtures produce these entries:
[users]
john_doe:
id: 1 # => primary keys are assigned to all entries
full_name: John Doe
jane_doe:
id: 2
full_name: Jane Doe
[groups]
admin_group:
id: 3
name: Administrators
worker_group:
id: 1
name: Workers
[user_groups]
admin_group_john_doe
id: 1
user_id: 1 # => references are resolved to their foreign keys
group_id: 3 # => explicitly set primary keys are respected
admin_group_jane_doe
id: 2
user_id: 2
group_id: 3
jane_doe_worker_group # => Notice the '<' operator in action
id: 3
user_id: 2
group_id: 1
Nesting is allowed. This will make the same entries as above:
[users.yml]
john_doe:
full_name: John Doe
groups:
admin_group:
id: 3
name: Administrators
users:
jane_doe:
full_name: Jane Doe
groups:
worker_group:
name: Workers
In this final form, JoinFix defines a highly-involved fixture in one chunk, in one file. This can be a BIG advantage when you try to test some cross-table, complicated lookup. The full fixture is centralized and easy to manage.
