pyrocore.torrent package

Torrent Backend Engines Package.

Copyright (c) 2010 The PyroScope Project <pyroscope.project@gmail.com>

Submodules

pyrocore.torrent.broom module

rTorrent Disk Space House-Keeping.

This is used in the rtsweep tool and the queue job of the pyrotoque daemon to free up disk space for new items, by deleting old items in a controlled way using a configurable order.

Copyright (c) 2018 The PyroScope Project <pyroscope.project@gmail.com>

class pyrocore.torrent.broom.DiskSpaceManager(config=None, rulesets=None)[source]

Bases: object

Core implementation of rtsweep.

class pyrocore.torrent.broom.SweepRule(ruleset, name, prio, order, filter)

Bases: tuple

filter

Alias for field number 4

name

Alias for field number 1

order

Alias for field number 3

prio

Alias for field number 2

ruleset

Alias for field number 0

pyrocore.torrent.broom.parse_cond(text)[source]

Parse a filter condition.

pyrocore.torrent.engine module

Torrent Engine Interface.

Copyright (c) 2009, 2010, 2011 The PyroScope Project <pyroscope.project@gmail.com>

class pyrocore.torrent.engine.ConstantField(valtype, name, doc, accessor=None, matcher=None, formatter=None, engine_name=None)[source]

Bases: pyrocore.torrent.engine.ImmutableField

Read-only download item field with constant value.

class pyrocore.torrent.engine.DynamicField(valtype, name, doc, accessor=None, matcher=None, formatter=None, engine_name=None)[source]

Bases: pyrocore.torrent.engine.ImmutableField

Read-only download item field with dynamic value.

class pyrocore.torrent.engine.FieldDefinition(valtype, name, doc, accessor=None, matcher=None, formatter=None, engine_name=None)[source]

Bases: object

Download item field.

FIELDS = {u'active': <DynamicField(<type 'int'>, u'active', u'last time a peer was connected')>, u'alias': <ConstantField(<function map_announce2alias at 0x7fe1c5bd8cf8>, u'alias', u'tracker alias or domain')>, u'completed': <DynamicField(<type 'int'>, u'completed', u'time download was finished')>, u'directory': <OnDemandField(<function to_unicode at 0x7fe1c543ac80>, u'directory', u'directory containing download data')>, u'done': <OnDemandField(<function percent at 0x7fe1c5b79b18>, u'done', u'completion in percent')>, u'down': <DynamicField(<type 'int'>, u'down', u'download rate')>, u'files': <OnDemandField(<type 'list'>, u'files', u'list of files in this item')>, u'fno': <OnDemandField(<type 'int'>, u'fno', u'number of files in this item')>, u'hash': <ConstantField(<type 'str'>, u'hash', u'info hash')>, u'is_active': <DynamicField(<type 'bool'>, u'is_active', u'download active?')>, u'is_complete': <DynamicField(<type 'bool'>, u'is_complete', u'download complete?')>, u'is_ghost': <DynamicField(<type 'bool'>, u'is_ghost', u'has no data file or directory?')>, u'is_ignored': <OnDemandField(<type 'bool'>, u'is_ignored', u'ignore commands?')>, u'is_multi_file': <OnDemandField(<type 'bool'>, u'is_multi_file', u'single- or multi-file download?')>, u'is_open': <DynamicField(<type 'bool'>, u'is_open', u'download open?')>, u'is_private': <ConstantField(<type 'bool'>, u'is_private', u'private flag set (no DHT/PEX)?')>, u'kind': <DynamicField(<type 'set'>, u'kind', u'ALL kinds of files in this item (the same as kind_0)')>, u'last_xfer': <DynamicField(<type 'int'>, u'last_xfer', u'last time data was transferred')>, u'leechtime': <DynamicField(<function untyped at 0x7fe1c5b79a28>, u'leechtime', u'time taken from start to completion')>, u'loaded': <DynamicField(<type 'int'>, u'loaded', u'time metafile was loaded')>, u'message': <OnDemandField(<function to_unicode at 0x7fe1c543ac80>, u'message', u'current tracker message')>, u'metafile': <ConstantField(<function to_unicode at 0x7fe1c543ac80>, u'metafile', u'path to torrent file')>, u'name': <ConstantField(<function to_unicode at 0x7fe1c543ac80>, u'name', u'name (file or root directory)')>, u'path': <DynamicField(<function to_unicode at 0x7fe1c543ac80>, u'path', u'path to download data')>, u'prio': <OnDemandField(<type 'int'>, u'prio', u'priority (0=off, 1=low, 2=normal, 3=high)')>, u'ratio': <DynamicField(<function ratio_float at 0x7fe1c5b79aa0>, u'ratio', u'normalized ratio (1:1 = 1.0)')>, u'realpath': <DynamicField(<function to_unicode at 0x7fe1c543ac80>, u'realpath', u'real path to download data')>, u'seedtime': <DynamicField(<function untyped at 0x7fe1c5b79a28>, u'seedtime', u'total seeding time after completion')>, u'sessionfile': <ConstantField(<function to_unicode at 0x7fe1c543ac80>, u'sessionfile', u'path to session file')>, u'size': <ConstantField(<type 'int'>, u'size', u'data size')>, u'started': <DynamicField(<type 'int'>, u'started', u'time download was FIRST started')>, u'stopped': <DynamicField(<type 'int'>, u'stopped', u'time download was last stopped or paused')>, u'tagged': <DynamicField(<type 'set'>, u'tagged', u"has certain tags? (not related to the 'tagged' view)")>, u'throttle': <OnDemandField(<type 'str'>, u'throttle', u'throttle group name (NULL=unlimited, NONE=global)')>, u'tracker': <ConstantField(<type 'str'>, u'tracker', u'first in the list of announce URLs')>, u'traits': <DynamicField(<type 'list'>, u'traits', u'automatic classification of this item (audio, video, tv, movie, etc.)')>, u'up': <DynamicField(<type 'int'>, u'up', u'upload rate')>, u'uploaded': <OnDemandField(<type 'int'>, u'uploaded', u'amount of uploaded data')>, u'views': <OnDemandField(<type 'set'>, u'views', u'views this item is attached to')>, u'xfer': <DynamicField(<type 'int'>, u'xfer', u'transfer rate')>}
classmethod lookup(name)[source]

Try to find field C{name}.

@return: Field descriptions, see C{matching.ConditionParser} for details.

class pyrocore.torrent.engine.ImmutableField(valtype, name, doc, accessor=None, matcher=None, formatter=None, engine_name=None)[source]

Bases: pyrocore.torrent.engine.FieldDefinition

Read-only download item field.

class pyrocore.torrent.engine.MutableField(valtype, name, doc, accessor=None, matcher=None, formatter=None, engine_name=None)[source]

Bases: pyrocore.torrent.engine.FieldDefinition

Writable download item field

class pyrocore.torrent.engine.OnDemandField(valtype, name, doc, accessor=None, matcher=None, formatter=None, engine_name=None)[source]

Bases: pyrocore.torrent.engine.DynamicField

Field that is fetched on first access only.

class pyrocore.torrent.engine.TorrentEngine[source]

Bases: object

A torrent backend.

group_by(fields, items=None)[source]

Returns a dict of lists of items, grouped by the given fields.

fields can be a string (one field) or an iterable of field names.

items(view=None, prefetch=None, cache=True)[source]

Get list of download items.

load_config(namespace=None, rcfile=None)[source]

Load engine configuration file.

log(msg)[source]

Log a message in the torrent client.

open()[source]

Open connection.

show(items, view=None)[source]

Visualize a set of items (search result), and return the view name.

view(viewname=u'default', matcher=None)[source]

Get list of download items.

class pyrocore.torrent.engine.TorrentProxy[source]

Bases: object

A single download item.

active

last time a peer was connected

classmethod add_custom_fields(*args, **kw)[source]

Add any custom fields defined in the configuration.

classmethod add_manifold_attribute(name)[source]

Register a manifold engine attribute.

@return: field definition object, or None if “name” isn’t a manifold attribute.

alias

tracker alias or domain

announce_urls(default=[])[source]

Get a list of all announce URLs.

completed

time download was finished

datapath()[source]

Get an item’s data path.

delete()[source]

Remove torrent from client.

directory

directory containing download data

done

completion in percent

down

download rate

fetch(name, engine_name=None)[source]

Get a field on demand.

“engine_name” is the internal name of the client engine.

files

list of files in this item

flush()[source]

Write volatile data to disk.

fno

number of files in this item

hash

info hash

hash_check()[source]

Hash check a download.

ignore(flag)[source]

Set ignore status.

is_active

download active?

is_complete

download complete?

is_ghost

Shining a light on the naming and paths mess:

hash=xxx for i in d.name d.base_filename d.base_path d.directory d.directory_base d.is_multi_file; do echo -n “$(printf ‘%20.20s ‘ $i)”; rtxmlrpc $i $hash done

Basics:
  • d.base_filename is always the basename of d.base_path
  • d.directory_base and d.directory are always the same
  • d.base_filename and d.base_path are empty on closed items, after a restart, i.e. not too useful (since 0.9.1 or so)
Behaviour of d.directory.set + d.directory_base.set (tested with 0.9.4):
  • d.base_path always remains unchanged, and item gets closed
  • d.start sets d.base_path if resume data ok
  • single:
    • d.directory[_base].set → d.name NEVER appended (only in d.base_path)
    • after start, d.base_path := d.directory/d.name
  • multi:
    • d.directory.set → d.name is appended
    • d.directory_base.set → d.name is NOT appended (i.e. item renamed to last path part)
    • after start, d.base_path := d.directory
Making sense of it (trying to at least):
  • d.directory is always a directory (thus, single items auto-append d.name in d.base_path and cannot be renamed)
  • d.directory_base.set means set path PLUS basename together for a multi item (thus allowing a rename)
  • only d.directory.set behaves consistently for single+multi, regarding the end result in d.base_path
is_ignored

ignore commands?

is_multi_file

single- or multi-file download?

is_open

download open?

is_private

private flag set (no DHT/PEX)?

kind

ALL kinds of files in this item (the same as kind_0)

last_xfer

last time data was transferred

leechtime

time taken from start to completion

loaded

time metafile was loaded

message

current tracker message

metafile

path to torrent file

name
path

path to download data

prio

priority (0=off, 1=low, 2=normal, 3=high)

ratio

normalized ratio (1:1 = 1.0)

realpath

real path to download data

seedtime

total seeding time after completion

sessionfile

path to session file

set_custom(key, value=None)[source]

Set a custom value. C{key} might have the form “key=value” when value is C{None}.

set_throttle(name)[source]

Assign to throttle group.

size

data size

start()[source]

(Re-)start downloading or seeding.

started

time download was FIRST started

stop()[source]

Stop and close download.

stopped

time download was last stopped or paused

tag(tags)[source]

Add or remove tags.

tagged

has certain tags? (not related to the ‘tagged’ view)

throttle

throttle group name (NULL=unlimited, NONE=global)

tracker

first in the list of announce URLs

traits

automatic classification of this item (audio, video, tv, movie, etc.)

up

upload rate

uploaded

amount of uploaded data

views

views this item is attached to

xfer

transfer rate

class pyrocore.torrent.engine.TorrentView(engine, viewname, matcher=None)[source]

Bases: object

A view on a subset of torrent items.

items()[source]

Get list of download items.

size()[source]

Total unfiltered size of view.

pyrocore.torrent.engine.detect_traits(item)[source]

Build traits list from attributes of the passed item. Currently, “kind_51”, “name” and “alias” are considered.

See pyrocore.util.traits:dectect_traits for more details.

pyrocore.torrent.engine.percent(floatval)[source]

Convert float ratio to a percent value.

pyrocore.torrent.engine.ratio_float(intval)[source]

Convert scaled integer ratio to a normalized float.

pyrocore.torrent.engine.untyped(val)[source]

A type specifier for fields that does nothing.

pyrocore.torrent.filter module

rTorrent Item Filter Jobs.

Copyright (c) 2012 The PyroScope Project <pyroscope.project@gmail.com>

class pyrocore.torrent.filter.ActionRule(config=None)[source]

Bases: pyrocore.torrent.filter.FilterJobBase

Perform an action on selected items.

run_filter(items)[source]

Perform configured action on filtered items.

class pyrocore.torrent.filter.FilterJobBase(config=None)[source]

Bases: object

Base class for filter rule jobs.

run()[source]

Filter job callback.

run_filter(items)[source]

Perform job on filtered items.

class pyrocore.torrent.filter.TorrentMirror(config=None)[source]

Bases: pyrocore.torrent.filter.FilterJobBase

Mirror selected items via a specified tracker.

run_filter(items)[source]

Load filtered items into remote client via tracker / watchdir.

pyrocore.torrent.formatting module

Torrent Item Formatting and Filter Rule Parsing.

Copyright (c) 2009, 2010, 2011 The PyroScope Project <pyroscope.project@gmail.com>

class pyrocore.torrent.formatting.OutputMapping(obj, defaults=None)[source]

Bases: pyrocore.util.algo.AttributeMapping

Map item fields for displaying them.

classmethod formatter_help()[source]

Return a list of format specifiers and their documentation.

pyrocore.torrent.formatting.expand_template(template, namespace)[source]

Expand the given (preparsed) template. Currently, only Tempita templates are supported.

@param template: The template, in preparsed form, or as a string (which then will be preparsed). @param namespace: Custom namespace that is added to the predefined defaults

and takes precedence over those.

@return: The expanded template. @raise LoggableError: In case of typical errors during template execution.

pyrocore.torrent.formatting.fmt_delta(timestamp)[source]

Format a UNIX timestamp to a delta (relative to now).

pyrocore.torrent.formatting.fmt_duration(duration)[source]

Format a duration value in seconds to a readable form.

pyrocore.torrent.formatting.fmt_iso(timestamp)[source]

Format a UNIX timestamp to an ISO datetime string.

pyrocore.torrent.formatting.fmt_json(val)[source]

JSON serialization.

pyrocore.torrent.formatting.fmt_mtime(val)[source]

Modification time of a path.

pyrocore.torrent.formatting.fmt_pathbase(val)[source]

Base name of a path.

pyrocore.torrent.formatting.fmt_pathdir(val)[source]

Directory containing the given path.

pyrocore.torrent.formatting.fmt_pathext(val)[source]

Extension of a path (including the ‘.’).

pyrocore.torrent.formatting.fmt_pathname(val)[source]

Base name of a path, without its extension.

pyrocore.torrent.formatting.fmt_pc(floatval)[source]

Scale a ratio value to percent.

pyrocore.torrent.formatting.fmt_strip(val)[source]

Strip leading and trailing whitespace.

pyrocore.torrent.formatting.fmt_subst(regex, subst)[source]

Replace regex with string.

pyrocore.torrent.formatting.fmt_sz(intval)[source]

Format a byte sized value.

pyrocore.torrent.formatting.format_item(format_spec, item, defaults=None)[source]

Format an item according to the given output format. The format can be gioven as either an interpolation string, or a Tempita template (which has to start with “E{lb}E{lb}”),

@param format_spec: The output format. @param item: The object, which is automatically wrapped for interpolation. @param defaults: Optional default values.

pyrocore.torrent.formatting.preparse(output_format)[source]

Do any special processing of a template, and return the result.

pyrocore.torrent.formatting.validate_field_list(fields, allow_fmt_specs=False, name_filter=None)[source]

Make sure the fields in the given list exist.

@param fields: List of fields (comma-/space-separated if a string). @type fields: list or str @return: validated field names. @rtype: list

pyrocore.torrent.formatting.validate_sort_fields(sort_fields)[source]

Make sure the fields in the given list exist, and return sorting key.

If field names are prefixed with ‘-‘, sort order is reversed for that field (descending).

pyrocore.torrent.jobs module

rTorrent Daemon Jobs.

Copyright (c) 2012 The PyroScope Project <pyroscope.project@gmail.com>

class pyrocore.torrent.jobs.EngineStats(config=None)[source]

Bases: object

rTorrent connection statistics logger.

run()[source]

Statistics logger job callback.

class pyrocore.torrent.jobs.InfluxDBStats(config=None)[source]

Bases: object

Push rTorrent and host statistics to InfluxDB.

run()[source]

Statistics feed job callback.

pyrocore.torrent.jobs.module_test()[source]

Quick test using…

python -m pyrocore.torrent.jobs

pyrocore.torrent.queue module

rTorrent Queue Manager.

Copyright (c) 2012 The PyroScope Project <pyroscope.project@gmail.com>

class pyrocore.torrent.queue.QueueManager(config=None)[source]

Bases: object

rTorrent queue manager implementation.

VIEWNAME = 'pyrotorque'
run()[source]

Queue manager job callback.

pyrocore.torrent.rtorrent module

rTorrent Proxy.

Copyright (c) 2009, 2010, 2011 The PyroScope Project <pyroscope.project@gmail.com>

class pyrocore.torrent.rtorrent.CommaLexer(text)[source]

Bases: shlex.shlex

Helper to split argument lists.

class pyrocore.torrent.rtorrent.RtorrentEngine[source]

Bases: pyrocore.torrent.engine.TorrentEngine

The rTorrent backend proxy.

CONSTANT_FIELDS = set(['tracker_size', 'name', 'is_multi_file', 'is_private', 'size_bytes', 'hash'])
CORE_FIELDS = set(['tied_to_file', 'tracker_size', 'name', 'size_bytes', 'is_private', 'hash', 'is_multi_file', 'complete'])
PREFETCH_FIELDS = set(['up_total', 'down_rate', 'tracker_size', 'name', 'ratio', 'custom=m_alias', 'is_active', 'custom=tm_loaded', 'base_path', 'tied_to_file', 'is_open', 'is_multi_file', 'down_total', 'size_bytes', 'custom=tm_completed', 'up_rate', 'custom=tm_started', 'hash', 'is_private', 'complete'])
PYRO2RT_MAPPING = {'custom_m_alias': 'custom=m_alias', 'custom_tm_completed': 'custom=tm_completed', 'custom_tm_loaded': 'custom=tm_loaded', 'custom_tm_started': 'custom=tm_started', 'down': 'down_rate', 'is_complete': 'complete', 'is_ignored': 'ignore_commands', 'metafile': 'tied_to_file', 'path': 'base_path', 'prio': 'priority', 'size': 'size_bytes', 'throttle': 'throttle_name', 'up': 'up_rate'}
RT2PYRO_MAPPING = {'base_path': 'path', 'complete': 'is_complete', 'custom=m_alias': 'custom_m_alias', 'custom=tm_completed': 'custom_tm_completed', 'custom=tm_loaded': 'custom_tm_loaded', 'custom=tm_started': 'custom_tm_started', 'down_rate': 'down', 'ignore_commands': 'is_ignored', 'priority': 'prio', 'size_bytes': 'size', 'throttle_name': 'throttle', 'tied_to_file': 'metafile', 'up_rate': 'up'}
RTORRENT_RC_ALIASES = {'network.scgi.open_local': 'scgi_local', 'network.scgi.open_port': 'scgi_port'}
RTORRENT_RC_KEYS = ('scgi_local', 'scgi_port')
item(infohash, prefetch=None, cache=False)[source]

Fetch a single item by its info hash.

items(view=None, prefetch=None, cache=True)[source]

Get list of download items.

@param view: Name of the view. @param prefetch: OPtional list of field names to fetch initially. @param cache: Cache items for the given view?

load_config(namespace=None, rcfile=None)[source]

Load file given in “rcfile”.

log(msg)[source]

Log a message in the torrent client.

multicall(viewname, fields)[source]

Query the given fields of items in the given view.

The result list contains named tuples, so you can access the fields directly by their name.

open()[source]

Open connection.

show(items, view=None, append=False, disjoin=False)[source]

Visualize a set of items (search result), and return the view name.

uptime

rTorrent’s uptime.

class pyrocore.torrent.rtorrent.RtorrentItem(engine_, fields)[source]

Bases: pyrocore.torrent.engine.TorrentProxy

A single download item.

announce_urls(default=[])[source]

Get a list of all announce URLs. Returns default if no trackers are found at all.

as_dict()[source]

Return known fields.

cull(file_filter=None, attrs=None)[source]

Delete ALL data files and remove torrent from client.

@param file_filter: Optional callable for selecting a subset of all files.
The callable gets a file item as described for RtorrentItem._get_files and must return True for items eligible for deletion.

@param attrs: Optional list of additional attributes to fetch for a filter.

datapath()[source]

Get an item’s data path.

delete()[source]

Remove torrent from client.

execute(commands)[source]

Execute XMLRPC command(s).

fetch(name, engine_name=None)[source]

Get a field on demand.

flush()[source]

Write volatile data to disk.

hash_check()[source]

Hash check a download.

ignore(flag)[source]

Set ignore status.

purge()[source]

Delete PARTIAL data files and remove torrent from client.

set_custom(key, value=None)[source]

Set a custom value. C{key} might have the form “key=value” when value is C{None}.

set_prio(prio)[source]

Set priority (0-3).

set_throttle(name)[source]

Assign to throttle group.

start()[source]

(Re-)start downloading or seeding.

stop()[source]

Stop and close download.

tag(tags)[source]

Add or remove tags.

pyrocore.torrent.rtorrent.run()[source]

Module level test.

pyrocore.torrent.watch module

rTorrent Watch Jobs.

Copyright (c) 2012 The PyroScope Project <pyroscope.project@gmail.com>

class pyrocore.torrent.watch.MetafileHandler(job, pathname)[source]

Bases: object

Handler for loading metafiles into rTorrent.

addinfo()[source]

Add known facts to templating namespace.

handle()[source]

Handle metafile.

load()[source]

Load metafile into client.

parse()[source]

Parse metafile and check pre-conditions.

class pyrocore.torrent.watch.RemoteWatch(config=None)[source]

Bases: object

rTorrent remote torrent file watch.

run()[source]

Check remote watch target.

class pyrocore.torrent.watch.TreeWatch(config=None)[source]

Bases: object

rTorrent folder tree watch via inotify.

run()[source]

Regular maintenance and fallback task.

setup()[source]

Set up inotify manager.

See https://github.com/seb-m/pyinotify/.

class pyrocore.torrent.watch.TreeWatchCommand[source]

Bases: pyrocore.scripts.base.ScriptBaseWithConfig

Use tree watcher directly from cmd line, call it like this:
python -m pyrocore.torrent.watch <DIR>

If the argument is a file, the templating namespace for that metafile is dumped (for testing and debugging purposes).

ARGS_HELP = '<directory>'
OPTIONAL_CFG_FILES = ['torque.ini']
STD_LOG_LEVEL = 10
classmethod main()[source]

The entry point.

mainloop()[source]

The main loop.

class pyrocore.torrent.watch.TreeWatchHandler(pevent=None, **kargs)[source]

Bases: pyinotify.ProcessEvent

inotify event handler for rTorrent folder tree watch.

See https://github.com/seb-m/pyinotify/.

METAFILE_EXT = ('.torrent', '.load', '.start', '.queue')
handle_path(event)[source]

Handle a path-related event.

my_init(**kw)[source]

This method is called from ProcessEvent.__init__(). This method is empty here and must be redefined to be useful. In effect, if you need to specifically initialize your subclass’ instance then you just have to override this method in your subclass. Then all the keyworded arguments passed to ProcessEvent.__init__() will be transmitted as parameters to this method. Beware you MUST pass keyword arguments though.

@param kargs: optional delegated arguments from __init__(). @type kargs: dict

process_IN_CLOSE_WRITE(event)[source]

File written.

process_IN_MOVED_TO(event)[source]

File moved into tree.

process_default(event)[source]

Fallback.