NAME

PipeC - manage command pipes

SYNOPSIS

  use lib '/proj/axaf/simul/lib/perl';
  use PipeC;

  my $pipe = new PipeC;

  $pipe->argsep( ' ' );

  $pipe->add( $command, $arg1, $arg2 );
  $pipe->add( $command, $arg1, { option => value } );
  my $cmd = $pipe->add( $command, $arg1, 
                     [ option1 => value1, option2 => value2] );

  $cmd->add( $arg1, $args, { option => value }, 
                  [option => value, option => value ] )

  $cmd->argsep( '=' );

  warn $pipe->dump, "\n";

  $pipe->run;

DESCRIPTION

PipeC provides a mechanism to maintain readable execution pipelines. Pipelines are created by adding commands to a PipeC object. Options to the commands are set using a readable format; PipeC takes care of quoting strings, sticking equal signs in, etc. The pipeline may be rendered into a nicely formatted string, as well as being executed (currently by the Bourne shell, /bin/sh).

PipeC actually manages a list of PipeC::Cmd objects. These objects also have methods; descriptions are available in PipeC::Cmd Methods

PipeC Methods

new [\%attr]

new creates a new PipeC object with the optionally specified attributes. The attributes are specified via a hash, which may have the following keys:

ArgSep: This specifies the default string to separate arguments from their values. It defaults to the = character. New commands inherit the current value of the ArgSep string. The default value may also be changed with the argsep method for PipeC objects. The separator for a given command may be changed with the argsep method for the command.

add( [\%attr,] $command, <arguments> )

This adds a new command to the PipeC object. It returns a handle to the command (which is itself a PipeC::Cmd object). The optional hash may be used to set attributes for the command. The \%attr hash may contain one of the following key/value pairs:

CmdSep: The string to print between commands. Defaults to `` \n''.
CmdPfx: The string to print before the command. It defaults to ``\t'' to line things up nicely.
CmdOptSep: The string to print between the command and the first option. Defaults to `` \\\n''.
OptPfx: The string to print before each option. Defaults to ``\t''.
OptSep: The string to print between the options. Defaults to `` \\\n''.
ArgSep: The argument separator. This defaults to separator in use at the time each command was created via the PipeC::add method.

Arguments to the command may be specified in one of the following ways:

As a simple string, e.g.,
```
        $pipe->add( 'ls', '-l' );
```
This option is useful for command options which do not take an argument.
As a reference to a hash, e.g.,
```
        $pipe->add( 'tar', { -f => 'foo.tar' } );
```
This method obviously is only useful for options which take an argument. Multiple options may be specified in the hash. Because it is a hash, the ordering of the options will not be kept as specified.
As a reference to an array, e.g.,
```
        $pipe->add( 'tar', [ -f => 'foo.tar' ] );
```
This method obviously is only useful for options which take an argument. Multiple options may be specified in the array. The ordering of options is retained.

The different methods of option specification may be mixed, e.g.,

        $pipe->add( 'tar',
                    '-v',
                    [ -f => 'foo.tar' ],
                    { -b => 100 } 
                  );

argsep( $argsep )

This sets the default PipeC string which separates options and their arguments. This affects subsequent commands added to the pipe only. The PipeC::Cmd::argsep method is available for existing commands.

stdin( $stdin )

This specifies a file to which the standard input stream of the pipeline will be connected. if $stdout is undef or unspecified, it cancels any value previously set.

stdout( $stdout )

This specifies a file to which the standard output stream of the pipeline will be written. if $stdout is undef or unspecified, it cancels any value previously set.

stderr( $stderr )

This specifies a file to which the standard output stream of the pipeline will be written. if $stderr is undef or unspecified, it cancels any value previously set.

stderr2stdout

This will redirect the standard error stream to the standard output stream. To cancel this, use stderr().

dump( \%attr )

This method returns a string containing the sequence of commands in the pipe. By default, this is a ``pretty'' representation. The \%attr hash may contain one of the following key/value pairs to change the output format:

CmdSep: The string to print between commands. Defaults to `` \n''.
CmdPfx: The string to print before the command. It defaults to ``\t'' to line things up nicely.
CmdOptSep: The string to print between the command and the first option. Defaults to `` \\\n''.
OptPfx: The string to print before each option. Defaults to ``\t''.
OptSep: The string to print between the options. Defaults to `` \\\n''.
ArgSep: The argument separator. This defaults to separator in use at the time each command was created via the PipeC::add method.

run

Execute the pipe. Currently this is done by passing it to the Perl system command and using the Bourne shell to evaluate the pipe. It returns the value returned by the system call.

valrep( $pattern, $value, [$lastvalue, [$firstvalue] )

Replace arguments to options whose arguments match the Perl regular expression, $pattern with $value. If $lastvalue is specified, the last matched argument will be replaced with $lastvalue. If $firstvalue is specified, the first matched argument will be replaced with $firstvalue.

For example,

        my $pipe = new PipeC;
        $pipe->add( 'cmd1', [ input => 'INPUT', output => 'OUTPUT' ] );
        $pipe->add( 'cmd2', [ input => 'INPUT', output => 'OUTPUT' ] );
        $pipe->add( 'cmd3', [ input => 'INPUT', output => 'OUTPUT' ] );
        $pipe->valrep( 'OUTPUT', 'stdout', 'output_file' );
        $pipe->valrep( 'INPUT', 'stdin', undef, 'input_file' );
        print $pipe->dump, "\n"

results in

                cmd1 \
                  input=input_file \
                  output=stdout \
        |       cmd2 \
                  input=stdin \
                  output=stdout \
        |       cmd3 \
                  input=stdin \
                  output=output_file

PipeC::Cmd Methods

PipeC::Cmd objects have methods of their own.

add( <arguments> )

This method adds additional arguments to the command. The format of the arguments is the same as to the PipeC::add method. This is useful if some arguments should be conditionally given, e.g.

        $cmd = $pipe->add( 'ls' );
        $cmd->add( '-l' ) if $want_long_listing;

dump( \%attr )

This method returns a string containing the command and its arguments. By default, this is a ``pretty'' representation. The \%attr hash may contain one of the following key/value pairs to change the output format:

CmdPfx: The string to print before the command. It defaults to ``\t'' to line things up nicely.
CmdOptSep: The string to print between the command and the first option. Defaults to `` \\\n''.
OptPfx: The string to print before each option. Defaults to ``\t''.
OptSep: The string to print between the options. Defaults to `` \\\n''.
ArgSep: The argument separator. This defaults to separator in use at the time each command was created via the PipeC::add method.

argsep( $argsep )

This specifies the string used to separate arguments from their values. It defaults to the default value for the parent PipeC object when the PipeC::Cmd object was created, which may be set via the PipeC::argsep method, or when the initial PipeC object is created.

valrep( $pattern, $value, [$lastvalue, [$firstvalue] ] )

Replace arguments to options whose arguments match the Perl regular expression, $pattern with $value. If $lastvalue is specified, the last matched argument will be replaced with $lastvalue.

For example,

        $cmd = $pipe->add( 'cmd1' );
        $cmd->add( [ opt1 => 'FOO' ] );
        $cmd->add( [ opt2 => 'FOO' ] );

        $cmd->valrep( 'FOO', 'FOO1', 'FOO2' );
        print $cmd->dump, "\n"

results in

                cmd1 \
                  opt1=FOO1 \
                  opt2=FOO2

If $firstvalue is specified, the first matched argument will be replaced with $lastvalue:

        $cmd = $pipe->add( 'cmd1' );
        $cmd->add( [ opt1 => 'FOO' ] );
        $cmd->add( [ opt2 => 'FOO' ] );

        $cmd->valrep( 'FOO', 'FOO1', undef, 'FOO2' );
        print $cmd->dump, "\n"

results in

                cmd1 \
                  opt1=FOO2 \
                  opt2=FOO1

Examples

Sometimes it's not possible to determine beforehand which command in a pipeline will be the final one in the pipe; thus, it's not possible to specify which command actually writes the output file until the very end. In the following example the programs recognize the token stdout to refer to the standard output stream; this is specific to their implementation.

        my $pipe = new PipeC;
        $pipe->add( 'genphot',
                    { output    => 'OUTPUT',
                      photdens  => 0.001 }
                  );

        $pipe->add( 'bp2rdb',
                    { input     => 'stdin',
                      output    => 'OUTPUT' }
                    )
                if $convert_to_rdb;

        $pipe->valrep( 'OUTPUT', 'stdout', 'rays.out' );

        print $pipe->dump, "\n"

This results in:

                genphot \
                  output=stdout \
                  photdens=0.001 \
        |       bp2rdb \
                  input=stdin \
                  output=rays.out

If programs can write to stdout directly, one can use PipeC::stdout (and PipeC::stderr, if need be):

        my $pipe = new PipeC;
        $pipe->add( 'ls' );
        $pipe->add( 'wc' );
        $pipe->stdout( 'line_count' );
        print $pipe->dump, "\n";

This results in:

                ls \
        |       wc \
        >       line_count

To redirect stderr, add the following line,

        $pipe->stderr( 'error' );

which results in:

        (       ls \
        |       wc \
        >       line_count \
        ) \
        2>      error

The entire pipe is run as a subshell, to ensure that all of the commands' standard error streams go to the same place.

LICENSE

This software is released under the GNU General Public License. You may find a copy at

   http://www.fsf.org/copyleft/gpl.html

AUTHOR

Diab Jerius (djerius@cfa.harvard.edu)