Bio::PullParserI - A base module for fast 'pull' parsing
# do not use this class, it is intended for parser module
# writers only
If you are writing a module to parse some new format, you may wish to use a
'pull' approach whereby you only do work (reading file data, parsing it,
turning the parsed data in an object) when absolutely necessary.
PullParserI provides a system for doing exactly that. As a PullParser you need a
chunk. A chunk is just a Bio::Root::IO that contains all the raw data you
would want to parse. You can use the
chunk() method to create a chunk
from a filename, existing filehandle or even a string. If you make a chunk
from a large file, but actually only want your chunk to be some portion of the
whole file, supply start and end amounts in bytes to
chunk() at the
same time. The methods
_chunk_seek() and
_chunk_tell() provide
seeks and tells that are relative to the start and end of your chunk, not the
whole file.
The other thing you will need to decide when making a chunk is how to handle
piped input. A PullParser typically needs seekable data to parse, so if your
data is piped in and unseekable, you must decide between creating a temp file
or reading the input into memory, which will be done before the chunk becomes
usable and you can begin any parsing. Alternatively you can choose to force a
sequential read, in which case you can make use of
_dependencies() to
define the linear order of methods that would result in the file being read
sequentially. The return value of
_sequential() is also useful here, if
you would need to cache some data or otherwise behave differently during a
sequential read.
The main method in the system is
get_field(). This method relies on the
existence of a private hash reference accessible to it with the method
_fields(). That hash ref should have as keys all the sorts of data you
will want to parse (eg. 'score'), and prior to parsing the values would be
undefined. A user of your module can then call either
$module->get_field('score') or $module->score and get_field will either
return the answer from $self->_fields->{score} if it is defined, or call
a method
_discover_score() first if not. So for the system to work you
need to define a _discover_*() method for every field in the fields hash, and
ensure that the method stores an answer in the fields hash.
How you implement your _discover_* methods is up to you, though you should never
call a _discover_* method directly yourself; always use
get_field(),
since
get_field() will deal with calling dependent methods for you if a
forced sequenctial read is in progress due to piped input. You will almost
certainly want to make use of the various chunk-related methods of this class
(that are denoted private by the leading '_'; this means you can use them as
the author of a parser class, but users of your parser should not).
Primary amongst them is _*
_chunk_by_end() to which you provide text that
represents the end of your desired chunk and it does a readline with your
argument as $/. The chunk knows about its line-endings, so if you want your
end definition to include a new line, just always use "\n" and
PullParserI will do any necessary conversion for you.
If your input data is hierarchical (eg. report->many results->many
hits->many hsps), and you want an object at the leaf of the hierarchy to
have access to information that is shared amongst all of them (is parsed in
the root), you don't have to copy the data to each leaf object; simply by
defining
parent(), when you call
get_field() and the requested
field isn't in your leaf's fields hash, the leaf's parent will be asked for
the field instead, and so on till root.
See Bio::SearchIO::hmmer_pull for an example of implementing a parser using
PullParserI.
User feedback is an integral part of the evolution of this and other Bioperl
modules. Send your comments and suggestions preferably to the Bioperl mailing
list. Your participation is much appreciated.
[email protected] - General discussion
http://bioperl.org/wiki/Mailing_lists - About the mailing lists
Please direct usage questions or support issues to the mailing list:
[email protected]
rather than to the module maintainer directly. Many experienced and reponsive
experts will be able look at the problem and quickly address it. Please
include a thorough description of the problem with code and data examples if
at all possible.
Report bugs to the Bioperl bug tracking system to help us keep track of the bugs
and their resolution. Bug reports can be submitted via the web:
https://github.com/bioperl/bioperl-live/issues
Email
[email protected]
Inspired by a posting by Aaron J. Mackey
The rest of the documentation details each of the object methods. Internal
methods are usually preceded with a _
Title : _fields
Usage : $obj->_fields( { field1 => undef } );
my $fields_ref = $obj->_fields;
Function: Get/set the hash reference containing all the fields for this parser
Returns : hash ref
Args : none to get, OR hash ref to set
Title : has_field
Usage : if ($obj->has_field('field_name') {...}
Function: Ask if a particular object has a given field (doesn't ask ancestors)
Returns : boolean
Args : string (the field name to test)
Title : get_field
Usage : my $field_value = $obj->get_field('field_name');
Function: Get the value of a given field. If this $obj doesn't have the field,
it's parent() will be asked, and so on until there are no more
parents.
Returns : scalar, warns if a value for the field couldn't be found and returns
undef.
Args : string (the field to get)
Title : parent
Usage : $obj->parent($parent_obj);
my $parent_obj = $obj->parent;
Function: Get/set the parent object of this one.
Returns : Bio::PullParserI
Args : none to get, OR Bio::PullParserI to set
Title : chunk
Usage : $obj->chunk($filename);
my $chunk = $obj->chunk;
Function: Get/set the chunk of this parser.
Returns : Bio:Root::IO
Args : none to get, OR
First argument of a GLOB reference, filename string, string data to
treat as the chunk, or Bio::Root::IO.
Optionally, also provide:
-start => int : the byte position within the thing described by the
first argument to consider as the start of this
chunk (default 0)
-end => int : the byte position to consider as the end (default
true end)
-piped_behaviour => 'memory'|'temp_file'|'sequential_read'
The last option comes into effect when the first argument is
something that cannot be seeked (eg. piped input filehandle).
'memory' means read all the piped input into a string
first, then set the chunk to that string.
'temp_file' means read all the piped input and output it to
a temp file, then set the chunk to that temp file.
'sequential_read' means that the piped input should be read
sequentially and your parsing code must cope with
not being able to seek.
'memory' is the fastest but uses the most memory. 'temp_file' and
'sequential_read' can be slow, with 'temp_file' being the most memory
efficient but requiring disc space. The default is 'sequential_read'.
Note that in versions of perl earlier than 5.8 only temp_file works
and will be used regardless of what value is supplied here.
Title : _sequential
Usage : if ($obj->_sequential) {...}
Function: Ask if we have to do operations such that the input is read
sequentially.
Returns : boolean
Args : none to get, OR boolean to set (typically, you should never set this
yourself)
Title : _dependencies
Usage : $obj->_dependencies( { field1 => field2 } );
my $dependancy = $obj->_dependencies('field_name');
Function: Set the fields that are dependent on each other, or get the field
than another is dependent upon.
Returns : string (a field name)
Args : string (a field name) to get, OR hash ref to initially set, with
field names as keys and values, key field being dependent upon value
field.
Title : _chunk_true_start
Usage : my $true_start = $obj->_chunk_true_start;
Function: Get/set the true start position of the chunk within the filehandle
it is part of.
Returns : int
Args : none to get, OR int to set (typically, you won't set this yourself)
Title : _chunk_true_end
Usage : my $true_end = $obj->_chunk_true_end;
Function: Get/set for the true end position of the chunk within the filehandle
it is part of.
Returns : int
Args : none to get, OR int to set (typically, you won't set this yourself)
Title : _line_ending
Usage : my $line_ending = $obj->_line_ending;
Function: Get/set for the line ending for the chunk.
Returns : string
Args : none to get, OR string to set (typically, you won't set this
yourself)
Title : _chunk_seek
Usage : $obj->_chunk_seek($pos);
Function: seek() the chunk to the provided position in bytes, relative to the
defined start of the chunk within its filehandle.
In _sequential() mode, this function does nothing.
Returns : n/a
Args : int
Title : _chunk_seek
Usage : my $pos = $obj->_chunk_tell;
Function: Get the current tell() position within the chunk, relative to the
defined start of the chunk within its filehandle.
In _sequential() mode, this function does nothing.
Returns : int
Args : none
Title : _chunk_seek
Usage : my $string = $obj->_get_chunk_by_nol;
Function: Get a chunk of chunk() from the current position onward for the given
number of lines.
Returns : string
Args : int (number of lines you want)
Title : _get_chunk_by_end
Usage : my $string = $obj->_get_chunk_by_end;
Function: Get a chunk of chunk() from the current position onward till the end
of the line, as defined by the supplied argument.
Returns : string
Args : string (line ending - if you want the line ending to include a new
line, always use \n)
Title : _find_chunk_by_end
Usage : my $string = $obj->_find_chunk_by_end;
Function: Get the start and end of what would be a chunk of chunk() from the
current position onward till the end of the line, as defined by the
supplied argument.
In _sequential() mode, this function does nothing.
Returns : _chunk_tell values for start and end in 2 element list
Args : string (line ending - if you want the line ending to include a new
line, always use \n)
Title : AUTOLOAD
Usage : n/a
Function: Assumes that any unknown method called should be treated as
get_field($method_name).
Returns : n/a
Args : n/a