Catmandu::Store::Solr

NAME

Catmandu::Store::Solr - A searchable store backed by Solr

SYNOPSIS

    # From the command line
    # Import data into Solr
    $ catmandu import JSON to Solr  < data.json
    # Export data from ElasticSearch
    $ catmandu export Solr to JSON > data.json
    # Export only one record
    $ catmandu export Solr --id 1234
    # Export using an Solr query
    $ catmandu export Solr --query "name:Recruitment OR name:college"
    # Export using a CQL query (needs a CQL mapping)
    $ catmandu export Solr --q "name any college"
    # From Perl
    use Catmandu::Store::Solr;
    my $store = Catmandu::Store::Solr->new(url => 'http://localhost:8983/solr' );
    my $obj1 = $store->bag->add({ name => 'Patrick' });
    printf "obj1 stored as %s\n" , $obj1->{_id};
    # Force an id in the store
    my $obj2 = $store->bag->add({ _id => 'test123' , name => 'Nicolas' });
    # send all changes to solr (committed automatically)
    $store->bag->commit;
    #transaction: rollback issued after 'die'
    $store->transaction(sub{
        $bag->delete_all();
        die("oops, didn't want to do that!");
    });
    my $obj3 = $store->bag->get('test123');
    $store->bag->delete('test123');
    $store->bag->delete_all;
    # All bags are iterators
    $store->bag->each(sub { ... });
    $store->bag->take(10)->each(sub { ... });
    # Search
    # Any extra arguments will be passed on as is to Solr
    my $hits = $store->bag->search(query => 'name:Patrick');

SOLR SCHEMA

The Solr schema needs to support at least the identifier field ("_id" by default) and a bag field ("_bag" by default) to be able to store Catmandu items:

    # In schema.xml
    <field name="_id"  type="string" indexed="true" stored="true" required="true" />
    <field name="_bag" type="string" indexed="true" stored="true" required="true" />

The names of these fields can optionally be changed using the "id_field" and "_bag" configuration parameters of Catmandu::Store::Solr. The "_id" will contain the record identifier. The "_bag" field will contain a string to support Catmandu::Bag-s in Solr.

CONFIGURATION

url

URL of Solr core Default: "http://localhost:8983/solr"

id_field

Name of unique field in Solr core. Default: "_id" This Solr field is mapped to "_id" when retrieved

bag_field

Name of field in Solr we can use to split the core into 'bags'. Default: "_bag" This Solr field is mapped to "_bag" when retrieved

on_error

Action to take when records cannot be saved to Solr. Default: throw. Available: ignore.

METHODS

new( url => $url )

new( url => $url, id_field => '_id', bag_field => '_bag' )

new( url => $url, bags => { data => { cql_mapping => \%mapping } } )

Creates a new Catmandu::Store::Solr store connected to a Solr core, specificied by $url. The store supports CQL searches when a cql_mapping is provided. This hash contains a translation of CQL fields into Solr searchable fields.

 # Example mapping
 $cql_mapping = {
      title => {
        op => {
          'any'   => 1 ,
          'all'   => 1 ,
          '='     => 1 ,
          '<>'    => 1 ,
          'exact' => {field => 'mytitle.exact' }
        } ,
        sort  => 1,
        field => 'mytitle',
        cb    => ['Biblio::Search', 'normalize_title']
      }
 }

The CQL mapping above will support for the 'title' field the CQL operators: any, all, =, <> and exact. For all the operators the 'title' field will be mapping into the Solr field 'mytitle', except for the 'exact' operator. In case of 'exact' we will search the field 'mytitle.exact'. The CQL has an optional callback field 'cb' which contains a reference to subroutines to rewrite or augment the search query. In this case, in the Biblio::Search package there is a normalize_title subroutine which returns a string or an ARRAY of string with augmented title(s). E.g.

    package Biblio::Search;
    sub normalize_title {
       my ($self,$title) = @_;
       my $new_title =~ s{[^A-Z0-9]+}{}g;
       $new_title;
    }
    1;

transaction

When you issue $bag->commit, all changes made in the buffer are sent to solr, along with a commit. So committing in Catmandu merely means flushing changes;-). When you wrap your subroutine within 'transaction', this behaviour is disabled temporarily. When you call 'die' within the subroutine, a rollback is sent to solr. Remember that transactions happen at store level: after the transaction, all buffers of all bags are flushed to solr, and a commit is issued in solr.

    # Record 'test' added
    $bag->add({ _id => "test" });
    # Buffer flushed, and 'commit' sent to solr
    $bag->commit();
    $bag->store->transaction(sub{
        $bag->add({ _id => "test",title => "test" });
        # Call to die: rollback sent to solr
        die("oops, didn't want to do that!");
    });
    # Record is still { _id => "test" }

INHERITED METHODS

This Catmandu::Store implements:

Catmandu::Store

Catmandu::Transactional

Each Catmandu::Bag in this Catmandu::Store implements:

Catmandu::Bag

Catmandu::Searchable

Catmandu::CQLSearchable

SEE ALSO

Catmandu::Store, WebService::Solr

AUTHOR

Nicolas Steenlant, "nicolas.steenlant at ugent.be" Patrick Hochstenbach, "patrick.hochstenbach at ugent.be" Nicolas Franck, "nicolas.franck at ugent.be" Pieter De Praetere

LICENSE AND COPYRIGHT

This program is free software; you can redistribute it and/or modify it under the terms of either: the GNU General Public License as published by the Free Software Foundation; or the Artistic License. See http://dev.perl.org/licenses/ for more information.