show me lines between two line markers / identifiers with perl

Start makers = 'begin - remote_properties' End markers = 'end - remote_properties'

perl -ne 'next unless /begin - remote_properties/ .. /end - remote_properties/;print' testfile.tibco_beGenerator

reading 3 lines in one time in oneliner perl

  cat PDDV_hosts.txt | perl -e 'while (my $frs=<>) { chomp $frs; $sec=<>; chomp $sec; $th=<> ;printf "%s,%s,%s", $frs, $sec, $th; }'

email all users that have exceeded their /home/ size quota with oneliner in du, perl

du -skh /home/* 2> /dev/null | sort -h | perl -pe 'if (/^\d{3}M/){ ($sz,$who) = $_=~ m!(\S+)\s\/home\/[a-z]{1,2}(75\d+)$!mxi;if ($who) {open(MAIL, "|/usr/sbin/sendmail -t");print MAIL "To: $who\@company.com\nFrom: mightymouse\@company.com\nSubject: homedir size exceeds: $sz; please remove items from your /home/ directory on uwb00078\n\nhomedir current size exceeds $sz > 99.9M; please remove items from your /home/ directory on uwb00078!";close(MAIL);print "$who $sz\n"}} $_=undef'

Perl find all modules on your INC path.

 perl -e 'foreach (@INC) {
    print `find $_ -type f -name "*.pm"`;
 }'

Remove arbitrary columns from text lines with perl and splice

Probably not the shorted ... remove the 9th and 0th column from an ls with perl and splice

find . -type f -ls | sort -k11 | perl -ne '@x=split;splice @x,9,1;splice @x,0,1; print join("\t", @x), "\n";' | tail 

List All Perl Modules That Are Installed On My Current System

One liner:

perl -MFile::Find=find -MFile::Spec::Functions -Tlwe 'find { wanted => sub { print canonpath $_ if /\.pm\z/ }, no_chdir => 1 }, @INC'

Perl: Exporting Static Methods with Moose and Sub::Exporter

Simple how to on exporting static metods with Moose and Sub::Exporter.

package Utils;

=head
   UTILS
=cut

use Moose;
use Carp qw/carp/;
use Sub::Exporter -setup => {
  exports => [
    qw(normalize bounds min max)
  ]
};

sub max {
  my ($value, $compare) = @_;
}
sub min {
  my ($value, $compare) = @_;
}
sub normalize {
  my ($value, $compare) = @_;
}

Dancer2 Session & Authentication and AngularJS

This took me too much to figure out from different sources hence jotting down my recipe. My application is called ADA ... here the main part with login / logout

package Ada;
use Dancer2;
use Dancer2::Session::Simple;
use Dancer2::Plugin::Auth::Tiny;

our $VERSION = '0.1';

# Authentication Requires Session, here the in memory
set session => "Simple";
#set session => 'YAML';

hook before => sub {
  printf "logged in? %s\n", session('user') ? session('user') : '-';
  if ( !session('user') && request->dispatch_path !~ m{^/login} ) {
    forward '/login', { requested_path => request->dispatch_path };
  }
};

get '/' => sub {
  template 'index';
};

get '/login' => sub {

  # Display a login page; the original URL they requested is available as
  # param('requested_path'), so could be put in a hidden field in the form
  template 'login', { path => param('requested_path') };
};

post '/login' => sub {
  if ( _is_valid( param('user'), param('pass') ) ) {
    print "logged on success!\n";
    session user => param('user');
    redirect param('path') || '/';
  }
  else {
    redirect '/login?failed=1';
  }
};

sub _is_valid {
  my ( $u, $p ) = @_;
  my $store = schema 'default';
  foreach my $r ( $store->resultset('PmiUser')->all ) {
    printf "%10s = %s\n", $r->fedex_id, $r->fedex_pw;
  }
  my $rec =
    $store->resultset('PmiUser')
    ->find( { 'id' => $u, 'pwd' => $p } );
  printf $rec->fedex_id;
  return $rec and $rec->fedex_id ? 0 : 1;
}

get '/logout' => sub {
  printf( "logging out %s\n", session('user') ? session('user') : '-' );
  context->destroy_session;
  redirect '/';
};

true;

project layout

.
├── Gruntfile.js
├── MANIFEST
├── MANIFEST.SKIP
├── Makefile.PL
├── app
│   ├── 404.html
│   ├── favicon.ico
│   ├── fonts
│   ├── images
│   │   └── yeoman.png
│   ├── index.html
│   ├── robots.txt
│   ├── scripts
│   │   ├── app.js
│   │   ├── controllers
│   │   │   ├── main.js
│   │   └── services
│   │       ├── lookups.js
│   ├── styles
│   │   └── main.css
│   ├── test
│   │   └── spec
│   │       └── services
│   │           └── lookups.js
│   └── views
│       └── main.html
├── bin
│   └── app.pl
├── bower.json
├── config.yml
├── environments
│   ├── development.yml
│   └── production.yml
├── karma-e2e.conf.js
├── karma.conf.js
├── lib
│   └── Ada.pm
├── out
├── package.json
├── run.sh
├── t
│   ├── 001_base.t
│   └── 002_index_route.t
├── test
│   ├── runner.html
│   └── spec
│       ├── controllers
│       │   ├── main.js
│       └── services
│           ├── lookups.js
│           └── wls-domain-service.js
└── views
    ├── index.tt -> ../app/index.html
    ├── layouts
    │   ├── main.tt -> ../../app/index.html
    │   └── navbar.tt
    └── login.tt

config.yml

# This is the main configuration file of your Dancer2 app
# env-related settings should go to environments/$env.yml
# all the settings in this file will be loaded at Dancer's startup.

# Your application's name
appname: "ada"

# when the charset is set to UTF-8 Dancer2 will handle for you
# all the magic of encoding and decoding. You should not care
# about unicode within your app when this setting is set (recommended).
charset: "UTF-8"

# template engine
# simple: default and very basic template engine
# template_toolkit: TT
serializer: JSON
session: YAML
session_dir: /tmp/dancer-sessions

template: "template_toolkit"
engines:
  JSON:
    allow_blessed:   '1'
    canonical:       '1'
    convert_blessed: '1' 
  template_toolkit:
    encoding:  'utf8'
    start_tag: '[%'
    end_tag:   '%]'
    WRAPPER: layouts/main.tt
    EVAL_PERL: '1'
  Auth::RBAC:
    credentials:
        class: Config
        options:
            accounts:
                  user01:
                      password: foobar
                      roles:
                          - guest
                          - user
                  weblogic:
                      password: wladmin1
                      roles:
                          - admin

angularjs - index.tt

Add the Logout function to your navigation pills.... make sure its a href.

  <!-- Add your site or application content here -->
  <div class="header">
    <ul class="nav nav-pills pull-right">
      <li class="active"><a ng-href="#">Home</a></li>
      <li><a href="./logout">Logout</a></li>
    </ul>
    <h3 class="text-muted">AD Assistant</h3>
  </div>

login.tt

A very simple ugly login screen.

<html>
      <head>
        <title>Session and logging in</title>
      </head>
      <body>
        <form action='/login' method='POST'>
            User Name : <input type='text' name='user'/>
            Password: <input type='password' name='pass' />

            <!-- Put the original path requested into a hidden
                       field so it's sent back in the POST and can be
                       used to redirect to the right page after login -->
            <input type='hidden' name='path' value='[% path %]'/>

            <input type='submit' value='Login' />
        </form>
      </body>
</html>

Dancer2 and changing server port

Well the --port does not work... keeps it on 3000 also set port does not work...

#!/usr/bin/env perl
use Dancer2;
set port => '3010';
use lib './lib';
use Data::Dumper;
dance;

does not work... What to do ??? Here the magic environment variables...

sub _build_default_config {
    my $self = shift;

    $ENV{PLACK_ENV}
      and $ENV{DANCER_APPHANDLER} = 'PSGI';

    return {
        apphandler   => ( $ENV{DANCER_APPHANDLER}   || 'Standalone' ),
        content_type => ( $ENV{DANCER_CONTENT_TYPE} || 'text/html' ),
        charset      => ( $ENV{DANCER_CHARSET}      || '' ),
        warnings     => ( $ENV{DANCER_WARNINGS}     || 0 ),
        startup_info => ( $ENV{DANCER_STARTUP_INFO} || 1 ),
        traces       => ( $ENV{DANCER_TRACES}       || 0 ),
        logger       => ( $ENV{DANCER_LOGGER}       || 'console' ),
        host         => ( $ENV{DANCER_SERVER}       || '0.0.0.0' ),
        port         => ( $ENV{DANCER_PORT}         || '3000' ),
        views        => ( $ENV{DANCER_VIEWS}
              || path( $self->config_location, 'views' ) ),
        appdir        => $self->location,
    };
}

So specify your bin/app.pl like this

#!/usr/bin/env perl

BEGIN {
  $ENV{'DANCER_PUBLIC'}            = './app';
  $ENV{'DBIC_TRACE'}               = 1;
  $ENV{'DANCER_PORT'}              = 3010;
  $ENV{'DBIC_NULLABLE_KEY_NOWARN'} = 1; # allowing nulls in uniq constraints
}
use Dancer2;
set port => '3010';
use lib './lib';
use myapp;
dance;

Dancer::Plugin::Pagination, AngularJS Pagination Abstraction and Integration

Out of the box pagination with Dancer or Dancer2 using AngularJS as front-end.... I just got sick of copy and paste these functionalities... Ingredients:

1. Dancer::Plugin::Pagination; class for implementing the pager() function in dancer with any DBIC class.
2. Object to JSON mapping; decoupling the DBIC ResultSet to a portable JSON struct.
3. Writing/Configuring the backend service for querying

At this point you should be able to get results back in the browser as a JSON resultset. Next is the AngularJS integration:

1. yo angular:route mytable; the controller and base class for ng-grid
2. yo angular:factory mytable; the service communicating with the Dancer backend
3. defining the pagination controller; abstract class with basic functions

1. Dancer::Plugin::Pagination

package Dancer::Plugin::Pagination;

use strict;
use warnings;
use Dancer2;
use Dancer2::Plugin;
use Dancer2::Logger::Console;
use Data::Dumper;

our $AUTHORITY         = 'KAAN';
our $VERSION           = '0.01';
our $DEFAULT_PAGE_SIZE = 5;
our $logger            = Dancer2::Logger::Console->new;

sub mapper {
  my ($self, $defs, @rs) = @_;
  return &map_fields ( $defs, @rs);
}

sub map_fields {
  my ( $field_defs, @rs ) = @_;

  #print "MAPPING:" . Dumper($field_defs);
  my @result = ();
  foreach my $row (@rs) {
    my $rec = {};
    while ( my ( $k, $v ) = each %$field_defs ) {
      if ( ref($v) eq 'ARRAY' ) {
        my $base = $row;
        foreach my $method (@$v) {

     #printf "%s ->[%s] %s\n", join('->', @$v) , $method, Dumper($row->columns);
          $base = $base->$method if $base;
        }
        $rec->{$k} = $base;
      }
      else {
        $rec->{$k} = $row->$v;
      }
    }
    push @result, $rec;
  }
  return \@result;

}

sub pagination {
  my ( $self, $store, $table, $mapping, $subselect ) = @_;
  unless ($mapping) {
    die( "missing mapping for " . $table );
    return 1;
  }

  # deserialize the filter, page and sort options.
  my $pager  = $self->from_json( $self->params->{'pager'} );
  my $filter = $self->from_json( $self->params->{'filter'} );
  my $sort   = $self->from_json( $self->params->{'sort'} );

  my $limit  = $pager  && $pager->{'pageSize'}    || $DEFAULT_PAGE_SIZE;
  my $pagenr = $pager  && $pager->{'currentPage'} || 1;
  my $type   = $filter && $filter->{'matchType'}  || 'any';
  my $text   = $filter && $filter->{'filterText'} || '';
  my $field         = $filter && $filter->{'matchFields'};
  my $sortFields    = $sort   && $sort->{'fields'};
  my $sortDirection = $sort   && $sort->{'directions'} || ['ASC'];
  # establish default filtering
  $field = { 'name' => 1 } unless keys %$field;
  #
  my %where = ();
  if ( $text ne '' ) {
    if ( $type =~ /all/ ) {
      foreach my $f ( keys %$field ) {
        $where{ $mapping->{$f} }{'-like'} = $text . '%';
      }
    }
    else {
      if ( scalar keys %$field > 1 ) {
        foreach my $f ( keys %$field ) {
          push @{ $where{'-or'} },
            { $mapping->{$f} => { '-like' => $text . '%' } };
        }
      }
      else {
        foreach my $f ( keys %$field ) {
          $where{ $mapping->{$f} }{'-like'} = $text . '%';
        }
      }
    }
  }
  my @order = ();
  @order = map { $mapping->{$_} . ' ASC' } @$sortFields if $sortFields;
  my @rs = $store->resultset(
                             $table)->search(
                                              \%where,
                                              {
                                                page     => $pagenr,
                                                rows     => $limit,
                                                order_by => \@order
                                              }
                             );
  my $count = $store->resultset($table)->search( \%where )->count;

  my $result = &map_fields( $mapping, @rs );
  return { 'totalItems' => $count, 'items' => $result };
}

register get_lookups => \&get_lookups;
register pager       => \&pagination;
register mapper      => \&mapper;
register_plugin for_versions => [ 1, 2 ];

1;

2. Mapping Object to JSON

In my case I want to map address information to a JSON object ... in my application package I add the following:

our %mapping = (
  'address' => {
    'id' => 'crc',
    'street' => 'street',
    'city' => 'city',
    'state' => 'state',
    'zip' => 'postal_code',
    'lname' => 'last_name',
    'fname' => 'first_name',
    'acc' => 'accuracy',
    'status' => [qw/status id/],  # many-to-one fetch status->id
    'ministry' => [qw/ministry_record id/] # many-to-one fetch ministry_record->id
  }
);

3. Adding the request with the pager functionality

The base request would be a simple definition of the schema, table name/resultset in DBIC and supplying the mapping.

=head
  ################################
=cut
get '/address/list' => sub {
  my $store = schema 'default';
  return pager($store, 'TerritoryAddress', $mapping{'address'})
};

Result

When we start the module we and enter http://localhost:3000/address/list we get the JSON result back. The default result set is a number of 5 with the total count of items in a separate field.

{"totalItems":"370","items":[{"street":"136 Morse Rd.","status":"ARC","fname":"Andy","state":"CT","acc":8,"city":"East Montpelier","zip":"XXX","lname":"Goodman","id":"1","ministry":"ARC"},{"street":"89 Maple St","status":"D","fname":"Diane","state":"VA","acc":8,"city":"SomewhereElse","zip":"XXX","lname":"ABCDE","id":"4","ministry":"RD"},{"street":"1409 North Ave","status":"H","fname":"Tim & Tina","state":"NL","acc":8,"city":"OceanDeep","zip":"XXX","lname":"ABCDE","id":"5","ministry":"RD"},{"street":"54 Roseade Parkway","status":"ARC","fname":null,"state":"NB","acc":8,"city":"AnotherPlace","zip":"XXX","lname":"ABCDE","id":"6","ministry":"ARC"},{"street":"714 Riverside Ave Apt 2","status":"HH","fname":"Dennis","state":"NC","acc":9,"city":"Somewhere","zip":"XXX","lname":null,"id":"7","ministry":"MVD"}]}

4. and 5. Adding the service and controller for AngularJS

Now essential are the pageOptions, filterOptions and sortOptions. These options will control the backend pager. If you notice we are extending from a base controller 'paginationControler' which houses the common setup of the pagination, thus avoiding duplication when more screens need the same functionality.

'use strict';

angular
  .module('MyApp')
  .controller(
    'AddressMgtCtrl',
    function($scope, $controller, $modal, $http, $timeout, addressMgt) {
        // Initialize the super class and extend it.
        $.extend(this, $controller('paginationControler', {$scope: $scope}));

     $scope.filterOptions.fields = [ {
      id : 'street',
      desc : 'Street'
     }, {
      id : 'city',
      desc : 'City'
     }, {
      id : 'fname',
      desc : 'First Name'
     } ];

     $scope.sortOptions = {
      fields : [ "street" ],
      directions : [ "ASC" ]
     };

     $scope.pageSync = function() {
      setTimeout(function() {
       addressMgt.search($scope.pagingOptions,
         $scope.filterOptions, $scope.sortOptions)
         .success($scope.responseFn);
      }, 100);
     };

     $scope.gridOptions = {
      data : 'data',
      enableRowSelection : true,
      enablePaging : true,
      enableCellEditOnFocus : true,
      showFilter : true,
      showGroupPanel : true,
      showFooter : true,
      filterOptions : $scope.filterOptions,
      pagingOptions : $scope.pagingOptions,
      sortInfo : $scope.sortOptions,
      multiSelect : false,
      selectedItems : $scope.mySelections,
      totalServerItems : 'totalItems',
      columnDefs : [ {
       field : 'street',
       displayName : 'Street',
       enableCellEdit : false
      }, {
       field : 'city',
       displayName : 'City',
       enableCellEdit : false
      }, {
       field : 'state',
       displayName : 'State',
       enableCellEdit : false
      }, {
       field : 'zip',
       displayName : 'Zip',
       enableCellEdit : false
      }, {
       field : 'lname',
       displayName : 'Last Name',
       enableCellEdit : false
      }, {
       field : 'fname',
       displayName : 'First Name',
       enableCellEdit : false
      } ]
     };

     $scope.pageSync();
    });

5. The Service definition

Our 'addressMgt' service will handle all of our communication for this screen and thus the pagination as well. You notice the page, filter and sort options are send as parameters to the pager service on the backend.

'use strict';
angular.module('MyApp').factory('addressMgt', function($http) {
 var addressAPI = {};
 addressAPI.search = function(pageOpts, filterOpts, sortOpts) {
  return $http.get('/address/list', {
   params : {
    'filter' : filterOpts && angular.toJson(filterOpts) || '',
    'pager' : pageOpts && angular.toJson(pageOpts) || '',
    'sort' : sortOpts && angular.toJson(sortOpts) || ''
   }
  });
 };

 return addressAPI;

});

6. Putting it all together

The 'views/address_mgt.html' now would be a one liner...

7. The Abstract Pagination Controller in Angular

The abstract 'paginationController' is the last piece that would make pagination somewhat abstract. I choose to define the controller in the 'app.js' layer as a central top layer function.

'use strict';

angular.module(
  'myApp',
  [ 'ngCookies', 'ngResource', 'ngSanitize', 'ngRoute', 'ngGrid',
    'ui.bootstrap' ]).config(function($routeProvider) {
 $routeProvider.when('/', {
  templateUrl : 'views/main.html',
  controller : 'MainCtrl'
 }).when('/address_mgt', {
  templateUrl : 'views/address_mgt.html',
  controller : 'AddressMgtCtrl'
 }).when('/address_dtl', {
...
...
... ADD THIS BELOW  ....
}).controller('paginationControler', function($scope) {
 /**
  * PAGINATION CONTROLLER
  */
 $scope.filterOptions = {
  useExternalFilter : true,
  filterText : "",
  show : false,
  matchType : 'any',
  matchFields : {
   'street' : true,
   'city' : true
  },
  fields : []
 };

 $scope.totalItems = 0;

 $scope.pagingOptions = {
  pageSizes : [ 25, 50, 100 ],
  pageSize : 5,
  currentPage : 1
 };

 $scope.responseFn = function(response) {
  console.log("%o", response);
  if (response) {
   $scope.data = response['items'];
   $scope.totalItems = response['totalItems'];
  } else {
   $scope.list = [];
   $scope.totalItems = 0;
  }
  if (!$scope.$$phase) {
   $scope.$apply();
  }
 };
 $scope.watchAttribute = function(newVal, oldVal) {
  if (newVal !== oldVal && newVal.currentPage !== oldVal.currentPage) {
   $scope.pageSync();
  }
 };
 $scope.$watch('pagingOptions.pageSize', function(oldVal, newVal) {
  if (newVal !== oldVal) {
   $scope.pagingOptions.currentPage = 1;
   $scope.pageSync();
  }
 }, true);

 $scope.$watch('pagingOptions', $scope.watchAttribute, true);

});

Dancer & Yeoman scaffolding... what needs changed

So here a quick one for integrating dancer and yeoman angular generation.

$ dancer -a myapp
$ cd myapp
$ yo angular myapp
$ rm public

# This is the main configuration file of your Dancer app
# env-related settings should go to environments/$env.yml
# all the settings in this file will be loaded at Dancer's startup.

# Your application's name
appname: "myapp"

# The default layout to use for your application (located in
# views/layouts/main.tt)
layout: "main"

# when the charset is set to UTF-8 Dancer will handle for you
# all the magic of encoding and decoding. You should not care
# about unicode within your app when this setting is set (recommended).
charset: "UTF-8"

# template engine
# simple: default and very basic template engine
# template_toolkit: 
serializer: JSON

template: "template_toolkit"
engines:
  JSON:
    allow_blessed:   '1'
    canonical:       '1'
    convert_blessed: '1' 
  template_toolkit:
    encoding:  'utf8'
    start_tag: '[%'
    end_tag:   '%]'
    WRAPPER: layouts/main.tt
    EVAL_PERL: '1'
 
public: app
auto_reload : true

# For session support enable the following line and see Dancer::Session
# session: "YAML"
plugins:
  DBIC:
    default:
      schema_class: Schema::PMI
      dsn: 'dbi:Oracle:sid=DDB00334;host=ddb00334.com;port=1526'
      user: 'XXXX'
      password: 'xxxx'
      on_connect_call: 'datetime_setup'

Dancer / Template Toolkit : auto include javascript sources

Playing with Dancer and Angular I found out spliting up controllers in separate js files seems a clean thing to do... however I kept on missing js includes... let's automate this in your 'layout/main.tt' insert the following to include the js pieces automagically

....

[% PERL %]
my @files = ();
use File::Find;
File::Find::find({ wanted=>sub {   push @files, $File::Find::name if /^.*\.js\z/s; }}, 'public/javascripts', 'public/partials');
foreach my $js (@files) {
    $js =~ s/public\///g;
    print "\n";
}
[% END %]


...
..

Then make sure you configure the template engine in your 'config.yml' for TT.

template: "template_toolkit"
engines:
  template_toolkit:
    encoding:  'utf8'
    start_tag: '[%'
    end_tag:   '%]'
    WRAPPER: layouts/main.tt
    EVAL_PERL: '1'

Then my source gets generated like this (lines 35-43) :

perl: Data::Dumper dyld: lazy symbol binding failed: Symbol not found: _isWORDCHAR Referenced from: /perl5/Library/Perl/Updates/5.12.3/darwin-thread-multi-2level/auto/Data/Dumper/Dumper.bundle Expected in: flat namespace dyld: Symbol not found: _isWORDCHAR

When installing Data::Dumper package 2.143 on MacOSX with perl 5.12.3 as runtime I received test results that were bad on the Dumper.xs tests. SO I forced the installation.... bad idea... got these errors on some of the Dumper calls when debugging DBIx result sets.

dyld: Symbol not found: _isWORDCHAR
  Referenced from: /Users/me/perl5/Library/Perl/Updates/5.12.3/darwin-thread-multi-2level/auto/Data/Dumper/Dumper.bundle
  Expected in: flat namespace

Solution: manually get out there on CPAN and get 2.145

$ cd .cpan/build
$ wget http://search.cpan.org/CPAN/authors/id/S/SM/SMUELLER/Data-Dumper-2.145.tar.gz
$ tar pfvx Data-Dumper-2.145.tar.gz
$ cd Data-Dumper-2.145
$ perl Makefile.PL
$ make && make test && make install

And it starts working ... here context of my Dumper calls that failed.... hope this helps anyone!

Can't set DBI::db=HASH(0x7f819f7c4108)->{HASH(0x7f819d3eab58)}: unrecognised attribute name or invalid value at /System/Library/Perl/Extras/5.12/darwin-thread-multi-2level/DBI.pm line 720.
Can't get DBI::db=HASH(0x7f819f7c4108)->{HASH(0x7f819d3eab58)}: unrecognised attribute name at /System/Library/Perl/Extras/5.12/darwin-thread-multi-2level/DBI.pm line 720.
SELECT me.server_os_type_map_seq, me.created_by_nm, me.created_dt, me.updated_by_nm, me.updated_dt, me.server_type_seq, me.os_version_seq FROM (
  SELECT me.server_os_type_map_seq, me.created_by_nm, me.created_dt, me.updated_by_nm, me.updated_dt, me.server_type_seq, me.os_version_seq  FROM PMI_SCHEMA.SERVER_OS_TYPE_MAP me 
) me WHERE ROWNUM <= ?
: '100'
SELECT me.os_version_seq, me.created_by_nm, me.created_dt, me.updated_by_nm, me.updated_dt, me.os_version_desc FROM PMI_SCHEMA.OS_VERSION me WHERE ( me.os_version_seq = ? ) : '32'
dyld: lazy symbol binding failed: Symbol not found: _isWORDCHAR
  Referenced from: /Users/me/perl5/Library/Perl/Updates/5.12.3/darwin-thread-multi-2level/auto/Data/Dumper/Dumper.bundle
  Expected in: flat namespace

dyld: Symbol not found: _isWORDCHAR
  Referenced from: /Users/me/perl5/Library/Perl/Updates/5.12.3/darwin-thread-multi-2level/auto/Data/Dumper/Dumper.bundle
  Expected in: flat namespace

Catalyst Schema generation for Oracle and specifying the dbi url notation

Getting connected to Oracle with Catalyst and generating the model and schema classes. What if you have a funky setup? Here the notation for host, port and sid locations on a public check (ie connect one user, schema is owned by another)

$ script/pmi_create.pl model PMI DBIC::Schema Schema::DBI create=static db_schema=GOG_SCHEMA 'dbi:Oracle:sid=DDB00334;host=ddb00334.google.com;port=1526' 'GOG_APP' 'MAG_APP'

At times explicitly defining the schema does not work. An alternate solution would be to define '%' as the schema (meaning any):

$  script/pmi_create.pl model PMI DBIC::Schema Schema::PMI create=static db_schema=% debug=true 'dbi:Oracle:sid=DDB00334;host=ddb00334.google.com;port=1526' 'GOG_APP' 'MAG_APP'

onliner - JAXBElement XJC Bindings and XMLRootElement creation for any schema

Some schema's do not get generated with the XMLRootElements which can get you into trouble marshalling Java 2 XML. See the error below

Caused by: javax.xml.bind.MarshalException
 - with linked exception:
[com.sun.istack.internal.SAXException2: unable to marshal type "com.tibco.xmlns.applicationmanagement.ApplicationType" as an element because it is missing an @XmlRootElement annotation]
 at com.sun.xml.internal.bind.v2.runtime.MarshallerImpl.write(MarshallerImpl.java:317)
 at com.sun.xml.internal.bind.v2.runtime.MarshallerImpl.marshal(MarshallerImpl.java:243)
 at org.springframework.oxm.jaxb.Jaxb2Marshaller.marshal(Jaxb2Marshaller.java:626)
 ... 25 more
Caused by: com.sun.istack.internal.SAXException2: unable to marshal type "com.tibco.xmlns.applicationmanagement.ApplicationType" as an element because it is missing an @XmlRootElement annotation
 at com.sun.xml.internal.bind.v2.runtime.XMLSerializer.reportError(XMLSerializer.java:216)
 at com.sun.xml.internal.bind.v2.runtime.ClassBeanInfoImpl.serializeRoot(ClassBeanInfoImpl.java:286)
 at com.sun.xml.internal.bind.v2.runtime.XMLSerializer.childAsRoot(XMLSerializer.java:462)
 at com.sun.xml.internal.bind.v2.runtime.MarshallerImpl.write(MarshallerImpl.java:314)
 ... 27 more
[ERROR] 

With the XJC bindings we can fix this however can be tedious going thru all the elements.... Boring.... so lets make a oneliner.... Here a one liner that will solve that and will generate the annotation section for your xcj.bindings or jaxb.bindings... Generating the XMLRootElement on bindings (requirements for perl will be XML::Simple see cpan)

perl -MXML::Simple -e 'BEGIN { use Data::Dumper; } $x=XMLin("./src/main/schema/5.5/ApplicationManagement.xsd");foreach (keys %{$x->{"complexType"}}) { printf "<jaxb:bindings node=\"xs:complexType[\@name=%c%s%c]\">
<annox:annotate><annox:annotate annox:class=\"javax.xml.bind.annotation.XmlRootElement\" name=\"%s\"/></annox:annotate></jaxb:bindings>\n",0x27,$_,0x27,$_ if $_; };'

Generates the output on the console:

<jaxb:bindings node="xs:complexType[=ActionType]"><annox:annotate><annox:annotate annox:class="javax.xml.bind.annotation.XmlRootElement" name="ActionType"/></annox:annotate></jaxb:bindings>
<jaxb:bindings node="xs:complexType[=ServiceType]"><annox:annotate><annox:annotate annox:class="javax.xml.bind.annotation.XmlRootElement" name="ServiceType"/></annox:annotate></jaxb:bindings>
<jaxb:bindings node="xs:complexType[=ServiceInstanceType]"><annox:annotate><annox:annotate annox:class="javax.xml.bind.annotation.XmlRootElement" name="ServiceInstanceType"/></annox:annotate></jaxb:bindings>
<jaxb:bindings node="xs:complexType[=anyObject]"><annox:annotate><annox:annotate annox:class="javax.xml.bind.annotation.XmlRootElement" name="anyObject"/></annox:annotate></jaxb:bindings>
<jaxb:bindings node="xs:complexType[=ApplicationType]"><annox:annotate><annox:annotate annox:class="javax.xml.bind.annotation.XmlRootElement" name="ApplicationType"/></annox:annotate></jaxb:bindings>
<jaxb:bindings node="xs:complexType[=NVPairType]"><annox:annotate><annox:annotate annox:class="javax.xml.bind.annotation.XmlRootElement" name="NVPairType"/></annox:annotate></jaxb:bindings>
<jaxb:bindings node="xs:complexType[=RepoInstanceType]"><annox:annotate><annox:annotate annox:class="javax.xml.bind.annotation.XmlRootElement" name="RepoInstanceType"/></annox:annotate></jaxb:bindings>
<jaxb:bindings node="xs:complexType[=RemoteRepoInstanceType]"><annox:annotate><annox:annotate annox:class="javax.xml.bind.annotation.XmlRootElement" name="RemoteRepoInstanceType"/></annox:annotate></jaxb:bindings>
<jaxb:bindings node="xs:complexType[=EventType]"><annox:annotate><annox:annotate annox:class="javax.xml.bind.annotation.XmlRootElement" name="EventType"/></annox:annotate></jaxb:bindings>
<jaxb:bindings node="xs:complexType[=BWServiceType]"><annox:annotate><annox:annotate annox:class="javax.xml.bind.annotation.XmlRootElement" name="BWServiceType"/></annox:annotate></jaxb:bindings>
Signings-MacBook-Pro:tibco-appmanage-schema $ 

THe end result as a XJC bindings file... which can be included in your XJC generation, in my case the maven plugin:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<jaxb:bindings xmlns:jaxb="http://java.sun.com/xml/ns/jaxb"
  xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xjc="http://java.sun.com/xml/ns/jaxb/xjc"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:annox="http://annox.dev.java.net"
  xsi:schemaLocation="http://java.sun.com/xml/ns/jaxb http://java.sun.com/xml/ns/jaxb/bindingschema_2_0.xsd"
  version="2.1">
  <jaxb:bindings schemaLocation="ApplicationManagement.xsd"
    node="/xs:schema">
    <jaxb:globalBindings generateIsSetMethod="true"
      collectionType="java.util.ArrayList">
    </jaxb:globalBindings>
    <jaxb:bindings node="xs:complexType[@name=ActionType]">
      <annox:annotate>
        <annox:annotate annox:class="javax.xml.bind.annotation.XmlRootElement"
          name="ActionType" />
      </annox:annotate>
    </jaxb:bindings>
    <jaxb:bindings node="xs:complexType[@name=ServiceType]">
      <annox:annotate>
        <annox:annotate annox:class="javax.xml.bind.annotation.XmlRootElement"
          name="ServiceType" />
      </annox:annotate>
    </jaxb:bindings>
    <jaxb:bindings node="xs:complexType[@name=ServiceInstanceType]">
      <annox:annotate>
        <annox:annotate annox:class="javax.xml.bind.annotation.XmlRootElement"
          name="ServiceInstanceType" />
      </annox:annotate>
    </jaxb:bindings>
    <jaxb:bindings node="xs:complexType[@name=anyObject]">
      <annox:annotate>
        <annox:annotate annox:class="javax.xml.bind.annotation.XmlRootElement"
          name="anyObject" />
      </annox:annotate>
    </jaxb:bindings>
    <jaxb:bindings node="xs:complexType[@name=ApplicationType]">
      <annox:annotate>
        <annox:annotate annox:class="javax.xml.bind.annotation.XmlRootElement"
          name="ApplicationType" />
      </annox:annotate>
    </jaxb:bindings>
    <jaxb:bindings node="xs:complexType[@name=NVPairType]">
      <annox:annotate>
        <annox:annotate annox:class="javax.xml.bind.annotation.XmlRootElement"
          name="NVPairType" />
      </annox:annotate>
    </jaxb:bindings>
    <jaxb:bindings node="xs:complexType[@name=RepoInstanceType]">
      <annox:annotate>
        <annox:annotate annox:class="javax.xml.bind.annotation.XmlRootElement"
          name="RepoInstanceType" />
      </annox:annotate>
    </jaxb:bindings>
    <jaxb:bindings node="xs:complexType[@name=RemoteRepoInstanceType]">
      <annox:annotate>
        <annox:annotate annox:class="javax.xml.bind.annotation.XmlRootElement"
          name="RemoteRepoInstanceType" />
      </annox:annotate>
    </jaxb:bindings>
    <jaxb:bindings node="xs:complexType[@name=EventType]">
      <annox:annotate>
        <annox:annotate annox:class="javax.xml.bind.annotation.XmlRootElement"
          name="EventType" />
      </annox:annotate>
    </jaxb:bindings>
    <jaxb:bindings node="xs:complexType[@name=BWServiceType]">
      <annox:annotate>
        <annox:annotate annox:class="javax.xml.bind.annotation.XmlRootElement"
          name="BWServiceType" />
      </annox:annotate>
    </jaxb:bindings>
  </jaxb:bindings>
</jaxb:bindings>

And the declaration of the maven plugin....

      <plugin>
        <groupId>org.jvnet.jaxb2.maven2</groupId>
        <artifactId>maven-jaxb2-plugin</artifactId>
        <version>0.8.3</version>
        <executions>
          <execution>
            <id>generate-domain1</id>
            <goals>
              <goal>generate</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <strict>true</strict>
          <schemaDirectory>src/main/schema/${schema.version}</schemaDirectory>
          <schemaIncludes>
            <value>scm-tibco-project.xsd</value>
          </schemaIncludes>
          <bindingIncludes>
            <include>xjc.bindings</include>
          </bindingIncludes>
          <verbose>true</verbose>
          <extension>true</extension>
          <args>
            <arg>-Xannotate</arg>
            <arg>-Xequals</arg>
            <arg>-XtoString</arg>
            <arg>-XhashCode</arg>
          </args>
          <generateDirectory>${project.build.directory}/generated-sources/xjc
          </generateDirectory>
          <plugins>
            <plugin>
              <groupId>org.jvnet.jaxb2_commons</groupId>
              <artifactId>jaxb2-basics</artifactId>
              <version>0.6.0</version>
            </plugin>
            <plugin>
              <groupId>org.jvnet.jaxb2_commons</groupId>
              <artifactId>jaxb2-basics-annotate</artifactId>
              <version>0.6.0</version>
            </plugin>
          </plugins>
        </configuration>
      </plugin>

Running perl system and cpan as non-root user...

The easiest method I have found so far is to say

wget -O- http://cpanmin.us | perl - -l ~/perl5 App::cpanminus local::lib

or for non wgetters..

curl -L http://cpanmin.us | perl - -l ~/perl5 App::cpanminus local::lib

Then:

eval `perl -I ~/perl5/lib/perl5 -Mlocal::lib`
echo 'eval `perl -I ~/perl5/lib/perl5 -Mlocal::lib`' >> ~/.profile

On Linux Systems

echo 'eval `perl -I ~/perl5/lib/perl5 -Mlocal::lib`' >> ~/.bash_profile

This assumes your profile is named .profile, you may need to change that to be .bash_profile, .bashrc, etc. After that you can install modules by saying

cpanm Module::Name

and simply use them the same way you would if the were installed in the root directories. In some cases the install does not work... then you can split things up (for the http_proxy it seems it explicitly needs 'http:' or 'https:'). The second item would add the ignore expired certificate on the wget command.

export http_proxy=http://my.proxy.org:3128
export https_proxy=http://my.proxy.org:3128
export PERL5LIB=~/perl5/lib/perl5/
wget --no-check-certificate -O cpan.pl http://cpanmin.us
perl cpan.pl -l ~/perl5 App::cpanminus local::lib

At times this even fails and use --force

perl cpan.pl --force -l ~/perl5 App::cpanminus local::lib

removing duplicate and empty elements from a perl array

Just another snippet not to remember.... removing the non-unique elements and the empty (null) elements.

sub uniq {
    return keys %{{ map { $_ => 1 } @_ }};
}

@my_array = ("one","two","three","two","three");
print join(" ", @my_array), "\n";
print join(" ", uniq(@my_array)), "\n";

using find and report on several lines with grep like functionality

Oneliner for finding a specific release of an artifact in a maven build file and reporting the artifact name and version even though these pieces are in different lines... And using find with the exit to print out only the successful files.

$ find . -name pom.xml -type f -exec perl -ne 'BEGIN {my $l} if (/2.0.1.RELEASE/) { printf "%s%s",$last,$_ ; $l=$last } ; $last=$_; END { exit 1 unless  $l; }' {} \; -print
   fmon-jms
   2.0.1.RELEASE
./master-service/master-service/master-proc/master-proc-core/pom.xml

find all the dependencies

usr@host:~/BUILD/foo-bar-root
$ find . -name 'pom.xml' -print -exec perl -ne 'BEGIN {my @k;} if (// ... /<\/plugin>/) { push @k, $_; } else { @k=undef;};  if (@k and grep {/<\/plugin>/} @k and grep {/pmd/} @k ) {print join("",@k);@k=undef}  ' {} \; -print 

Capture Contents from Match until the end of the file - in perl

I want the classpath that is set in the manifest.mf

Class-Path:  mscorlib.jar Microsoft.VisualBasic.jar rdVbScriptTranslat
 or.jar js.jar rdsuperscript.jar System.Xml.jar rt.jar System.jar Syst
 em.Data.jar System.Web.jar rt.jar J2SE.Helpers.jar rt.jar fop.jar

The whole file is

$ cat META-INF/MANIFEST.MF 
Manifest-Version: 1.0
VMW-Assembly: c%3A%5CReportDev%5CVB%5CrdXslFoTemplate%5Cbin%5CDebug_Ja
 va%5CrdXslFoTemplate.dll
Created-By: MainSoft Inc
Version-Date: Mon 02/20/06 15:49:54.479
Assembly-Version: 1.0.0.0
Build-Version: 2.2.0.130
Class-Path:  mscorlib.jar Microsoft.VisualBasic.jar rdVbScriptTranslat
 or.jar js.jar rdsuperscript.jar System.Xml.jar rt.jar System.jar Syst
 em.Data.jar System.Web.jar rt.jar J2SE.Helpers.jar rt.jar fop.jar

$ 

What to do.... make the 3 dot match and make the last match \cD which is code for end of file....

$ find . -name 'rdXsl*.jar' -type f | while read dir; do [ -f META-INF/MANIFEST.MF ] && rm -f META-INF/MANIFEST.MF;unzip -q $dir META-INF/MANIFEST.MF; cat META-INF/MANIFEST.MF | perl -ne 'print if /^Class-Path/ ... /\cD/' ; done;