Exscript package

The core module.

class Exscript.FileLogger(logdir, mode=u'a', delete=False, clearmem=True)[source]

Bases: Exscript.logger.Logger

A Logger that stores logs into files.

__init__(logdir, mode=u'a', delete=False, clearmem=True)[source]

The logdir argument specifies the location where the logs are stored. The mode specifies whether to append the existing logs (if any). If delete is True, the logs are deleted after they are completed, unless they have an error in them. If clearmem is True, the logger does not store a reference to the log in it. If you want to use the functions from Exscript.util.report with the logger, clearmem must be False.

add_log(job_id, name, attempt)[source]
log_aborted(job_id, exc_info)[source]
log_succeeded(job_id)[source]
class Exscript.Logger[source]

Bases: future.types.newobject.newobject

A QueueListener that implements logging for the queue. Logs are kept in memory, and not written to the disk.

__init__()[source]

Creates a new logger instance. Use the Exscript.util.log.log_to decorator to send messages to the logger.

add_log(job_id, name, attempt)[source]
get_aborted_actions()[source]

Returns the number of jobs that were aborted.

get_aborted_logs()[source]
get_logs()[source]
get_succeeded_actions()[source]

Returns the number of jobs that were completed successfully.

get_succeeded_logs()[source]
log(job_id, message)[source]
log_aborted(job_id, exc_info)[source]
log_succeeded(job_id)[source]
class Exscript.Host(uri, default_protocol='telnet')[source]

Bases: future.types.newobject.newobject

Represents a device on which to open a connection.

__init__(uri, default_protocol='telnet')[source]

Constructor. The given uri is passed to Host.set_uri(). The default_protocol argument defines the protocol that is used in case the given uri does not specify it.

Parameters:
  • uri (string) – A hostname; see set_uri() for more info.
  • default_protocol (string) – The protocol name.
account
address
append(name, value)[source]

Appends the given value to the list variable with the given name.

Parameters:
  • name (string) – The name of the variable.
  • value (object) – The appended value.
get(name, default=None)[source]

Returns the value of the given variable, or the given default value if the variable is not defined.

Parameters:
  • name (string) – The name of the variable.
  • default (object) – The default value.
Return type:

object

Returns:

The value of the variable.

get_account()[source]

Returns the account that is used to log in.

Return type:Account
Returns:The account.
get_address()[source]

Returns the address that is used to open the connection.

Return type:string
Returns:The address that is used to open the connection.
get_all()[source]

Returns a dictionary containing all variables.

Return type:dict
Returns:The dictionary with the variables.
get_dict()[source]

Returns a dict containing the host’s attributes. The following keys are contained:

  • hostname
  • address
  • protocol
  • port
Return type:dict
Returns:The resulting dictionary.
get_name()[source]

Returns the name.

Return type:string
Returns:The hostname excluding the name.
get_option(name, default=None)[source]

Returns the value of the given option if it is defined, returns the given default value otherwise.

Parameters:
  • name (str) – The option name.
  • default (object) – A default value.
get_options()[source]

Return a dictionary containing all defined options.

Return type:dict
Returns:The options.
get_protocol()[source]

Returns the name of the protocol.

Return type:string
Returns:The protocol name.
get_tcp_port()[source]

Returns the TCP port number.

Return type:string
Returns:The TCP port number.
get_uri()[source]

Returns a URI formatted representation of the host, including all of it’s attributes except for the name. Uses the address, not the name of the host to build the URI.

Return type:str
Returns:A URI.
has_key(name)[source]

Returns True if the variable with the given name is defined, False otherwise.

Parameters:name (string) – The name of the variable.
Return type:bool
Returns:Whether the variable is defined.
name
options
protocol
set(name, value)[source]

Stores the given variable/value in the object for later retrieval.

Parameters:
  • name (string) – The name of the variable.
  • value (object) – The value of the variable.
set_account(account)[source]

Defines the account that is used to log in.

Parameters:account (Exscript.Account) – The account.
set_address(address)[source]

Set the address of the remote host the is contacted, without changing hostname, username, password, protocol, and TCP port number. This is the actual address that is used to open the connection.

Parameters:address (string) – A hostname or IP name.
set_all(variables)[source]

Like set(), but replaces all variables by using the given dictionary. In other words, passing an empty dictionary results in all variables being removed.

Parameters:variables (dict) – The dictionary with the variables.
set_default(name, value)[source]

Like set(), but only sets the value if the variable is not already defined.

Parameters:
  • name (string) – The name of the variable.
  • value (object) – The value of the variable.
set_name(name)[source]

Set the hostname of the remote host without changing username, password, protocol, and TCP port number.

Parameters:name (string) – A hostname or IP address.
set_option(name, value)[source]

Defines a (possibly protocol-specific) option for the host. Possible options include:

verify_fingerprint: bool
Parameters:
  • name (str) – The option name.
  • value (object) – The option value.
set_protocol(protocol)[source]

Defines the protocol. The following protocols are currently supported:

  • telnet: Telnet
  • ssh1: SSH version 1
  • ssh2: SSH version 2
  • ssh: Automatically selects the best supported SSH version
  • dummy: A virtual device that accepts any command, but that does not respond anything useful.
  • pseudo: A virtual device that loads a file with the given “hostname”. The given file is a Python file containing information on how the virtual device shall respond to commands. For more information please refer to the documentation of protocols.Dummy.load_command_handler_from_file().
Parameters:protocol (string) – The protocol name.
set_tcp_port(tcp_port)[source]

Defines the TCP port number.

Parameters:tcp_port (int) – The TCP port number.
set_uri(uri)[source]

Defines the protocol, hostname/address, TCP port number, username, and password from the given URL. The hostname may be URL formatted, so the following formats are all valid:

For a list of supported protocols please see set_protocol().

Parameters:uri (string) – An URL formatted hostname.
tcp_port
vars
class Exscript.Account(name, password='', password2=None, key=None, needs_lock=True)[source]

Bases: future.types.newobject.newobject

This class represents a user account.

__init__(name, password='', password2=None, key=None, needs_lock=True)[source]

Constructor.

The authorization password is only required on hosts that separate the authentication from the authorization procedure. If an authorization password is not given, it defaults to the same value as the authentication password.

If the needs_lock argument is set to True, we ensure that no two threads can use the same account at the same time. You will want to use this setting if you are using a central authentication server that allows for only one login to happen at a time. Note that you will still be able to open multiple sessions at the time. It is only the authentication procedure that will not happen in parallel; once the login is complete, other threads can use the account again. In other words, the account is only locked during calls to protocols.Protocol.login() and the *authenticate* methods.

Warning

Setting lock to True drastically degrades performance!

Parameters:
  • name (str) – A username.
  • password (str) – The authentication password.
  • password2 (str) – The authorization password, if required.
  • key (PrivateKey) – A private key, if required.
  • needs_lock (bool) – True if the account will be locked during login.
acquire(signal=True)[source]

Locks the account. Method has no effect if the constructor argument needs_lock wsa set to False.

Parameters:signal (bool) – Whether to emit the acquired_event signal.
context()[source]

When you need a ‘with’ context for an already-acquired account.

get_authorization_password()[source]

Returns the authorization password of the account.

Return type:string
Returns:The account password.
get_key()[source]

Returns the key of the account, if any.

Return type:PrivateKey|None
Returns:A key object.
get_name()[source]

Returns the name of the account.

Return type:string
Returns:The account name.
get_password()[source]

Returns the password of the account.

Return type:string
Returns:The account password.
release(signal=True)[source]

Unlocks the account. Method has no effect if the constructor argument needs_lock wsa set to False.

Parameters:signal (bool) – Whether to emit the released_event signal.
set_authorization_password(password)[source]

Changes the authorization password of the account.

Parameters:password (string) – The new authorization password.
set_name(name)[source]

Changes the name of the account.

Parameters:name (string) – The account name.
set_password(password)[source]

Changes the password of the account.

Parameters:password (string) – The account password.
class Exscript.AccountPool(accounts=None)[source]

Bases: future.types.newobject.newobject

This class manages a collection of available accounts.

__init__(accounts=None)[source]

Constructor.

Parameters:accounts (Account|list[Account]) – Passed to add_account()
acquire_account(account=None, owner=None)[source]

Waits until an account becomes available, then locks and returns it. If an account is not passed, the next available account is returned.

Parameters:
  • account (Account) – The account to be acquired, or None.
  • owner (object) – An optional descriptor for the owner.
Return type:

Account

Returns:

The account that was acquired.

add_account(accounts)[source]

Adds one or more account instances to the pool.

Parameters:accounts (Account|list[Account]) – The account to be added.
get_account_from_hash(account_hash)[source]

Returns the account with the given hash, or None if no such account is included in the account pool.

get_account_from_name(name)[source]

Returns the account with the given name.

Parameters:name (string) – The name of the account.
has_account(account)[source]

Returns True if the given account exists in the pool, returns False otherwise.

Parameters:account (Account) – The account object.
n_accounts()[source]

Returns the number of accounts that are currently in the pool.

release_accounts(owner)[source]

Releases all accounts that were acquired by the given owner.

Parameters:owner (object) – The owner descriptor as passed to acquire_account().
reset()[source]

Removes all accounts.

class Exscript.Queue(domain='', verbose=1, mode='threading', max_threads=1, host_driver=None, exc_cb=None, stdout=<open file '<stdout>', mode 'w'>, stderr=<open file '<stderr>', mode 'w'>)[source]

Bases: future.types.newobject.newobject

Manages hosts/tasks, accounts, connections, and threads.

__init__(domain='', verbose=1, mode='threading', max_threads=1, host_driver=None, exc_cb=None, stdout=<open file '<stdout>', mode 'w'>, stderr=<open file '<stderr>', mode 'w'>)[source]

Constructor. All arguments should be passed as keyword arguments. Depending on the verbosity level, the following types of output are written to stdout/stderr (or to whatever else is passed in the stdout/stderr arguments):

  • S = status bar
  • L = live conversation
  • D = debug messages
  • E = errors
  • ! = errors with tracebacks
  • F = fatal errors with tracebacks

The output types are mapped depending on the verbosity as follows:

  • verbose = -1: stdout = None, stderr = F
  • verbose = 0: stdout = None, stderr = EF
  • verbose = 1, max_threads = 1: stdout = L, stderr = EF
  • verbose = 1, max_threads = n: stdout = S, stderr = EF
  • verbose >= 2, max_threads = 1: stdout = DL, stderr = !F
  • verbose >= 2, max_threads = n: stdout = DS, stderr = !F
Parameters:
  • domain (str) – The default domain of the contacted hosts.
  • verbose (int) – The verbosity level.
  • mode (str) – ‘multiprocessing’ or ‘threading’
  • max_threads (int) – The maximum number of concurrent threads.
  • host_driver (str) – driver name like “ios” for manual override
  • exc_cb (func(jobname, exc_info)) – callback function to call on exceptions
  • stdout (file) – The output channel, defaults to sys.stdout.
  • stderr (file) – The error channel, defaults to sys.stderr.
add_account(account)[source]

Adds the given account to the default account pool that Exscript uses to log into all hosts that have no specific Account attached.

Parameters:account (Account) – The account that is added.
add_account_pool(pool, match=None)[source]

Adds a new account pool. If the given match argument is None, the pool the default pool. Otherwise, the match argument is a callback function that is invoked to decide whether or not the given pool should be used for a host.

When Exscript logs into a host, the account is chosen in the following order:

# Exscript checks whether an account was attached to the Host object using Host.set_account()), and uses that.

# If the Host has no account attached, Exscript walks through all pools that were passed to Queue.add_account_pool(). For each pool, it passes the Host to the function in the given match argument. If the return value is True, the account pool is used to acquire an account. (Accounts within each pool are taken in a round-robin fashion.)

# If no matching account pool is found, an account is taken from the default account pool.

# Finally, if all that fails and the default account pool contains no accounts, an error is raised.

Example usage:

def do_nothing(conn):
    conn.autoinit()

def use_this_pool(host):
    return host.get_name().startswith('foo')

default_pool = AccountPool()
default_pool.add_account(Account('default-user', 'password'))

other_pool = AccountPool()
other_pool.add_account(Account('user', 'password'))

queue = Queue()
queue.add_account_pool(default_pool)
queue.add_account_pool(other_pool, use_this_pool)

host = Host('localhost')
queue.run(host, do_nothing)

In the example code, the host has no account attached. As a result, the queue checks whether use_this_pool() returns True. Because the hostname does not start with ‘foo’, the function returns False, and Exscript takes the ‘default-user’ account from the default pool.

Parameters:
  • pool (AccountPool) – The account pool that is added.
  • match (callable) – A callback to check if the pool should be used.
destroy(force=False)[source]

Like shutdown(), but also removes all accounts, hosts, etc., and does not restart the queue. In other words, the queue can no longer be used after calling this method.

Parameters:force (bool) – Whether to wait until all jobs were processed.
enqueue(function, name=None, attempts=1)[source]

Places the given function in the queue and calls it as soon as a thread is available. To pass additional arguments to the callback, use Python’s functools.partial().

Parameters:
  • function (function) – The function to execute.
  • name (string) – A name for the task.
  • attempts (int) – The number of attempts on failure.
Return type:

object

Returns:

An object representing the task.

force_run(hosts, function, attempts=1)[source]

Like priority_run(), but starts the task immediately even if that max_threads is exceeded.

Parameters:
  • hosts (string|list(string)|Host|list(Host)) – A hostname or Host object, or a list of them.
  • function (function) – The function to execute.
  • attempts (int) – The number of attempts on failure.
Return type:

object

Returns:

An object representing the task.

get_max_threads()[source]

Returns the maximum number of concurrent threads.

Return type:int
Returns:The maximum number of connections.
get_progress()[source]

Returns the progress in percent.

Return type:float
Returns:The progress in percent.
is_completed()[source]

Returns True if the task is completed, False otherwise. In other words, this methods returns True if the queue is empty.

Return type:bool
Returns:Whether all tasks are completed.
join()[source]

Waits until all jobs are completed.

priority_run(hosts, function, attempts=1)[source]

Like run(), but adds the task to the front of the queue.

Parameters:
  • hosts (string|list(string)|Host|list(Host)) – A hostname or Host object, or a list of them.
  • function (function) – The function to execute.
  • attempts (int) – The number of attempts on failure.
Return type:

object

Returns:

An object representing the task.

priority_run_or_raise(hosts, function, attempts=1)[source]

Like priority_run(), but if a host is already in the queue, the existing host is moved to the top of the queue instead of enqueuing the new one.

Parameters:
  • hosts (string|list(string)|Host|list(Host)) – A hostname or Host object, or a list of them.
  • function (function) – The function to execute.
  • attempts (int) – The number of attempts on failure.
Return type:

object

Returns:

A task object, or None if all hosts were duplicates.

reset()[source]

Remove all accounts, hosts, etc.

run(hosts, function, attempts=1)[source]

Add the given function to a queue, and call it once for each host according to the threading options. Use decorators.bind() if you also want to pass additional arguments to the callback function.

Returns an object that represents the queued task, and that may be passed to is_completed() to check the status.

Parameters:
  • hosts (string|list(string)|Host|list(Host)) – A hostname or Host object, or a list of them.
  • function (function) – The function to execute.
  • attempts (int) – The number of attempts on failure.
Return type:

object

Returns:

An object representing the task.

run_or_ignore(hosts, function, attempts=1)[source]

Like run(), but only appends hosts that are not already in the queue.

Parameters:
  • hosts (string|list(string)|Host|list(Host)) – A hostname or Host object, or a list of them.
  • function (function) – The function to execute.
  • attempts (int) – The number of attempts on failure.
Return type:

object

Returns:

A task object, or None if all hosts were duplicates.

set_max_threads(n_connections)[source]

Sets the maximum number of concurrent connections.

Parameters:n_connections (int) – The maximum number of connections.
shutdown(force=False)[source]

Stop executing any further jobs. If the force argument is True, the function does not wait until any queued jobs are completed but stops immediately.

After emptying the queue it is restarted, so you may still call run() after using this method.

Parameters:force (bool) – Whether to wait until all jobs were processed.
class Exscript.PrivateKey(keytype='rsa')[source]

Bases: future.types.newobject.newobject

Represents a cryptographic key, and may be used to authenticate useing Exscript.protocols.

__init__(keytype='rsa')[source]

Constructor. Supported key types are provided by their respective protocol adapters and can be retrieved from the PrivateKey.keytypes class attribute.

Parameters:keytype (string) – The key type.
static from_file(filename, password='', keytype=None)[source]

Returns a new PrivateKey instance with the given attributes. If keytype is None, we attempt to automatically detect the type.

Parameters:
  • filename (string) – The key file name.
  • password (string) – The key password.
  • keytype (string) – The key type.
Return type:

PrivateKey

Returns:

The new key.

get_filename()[source]

Returns the name of the key file.

Return type:string
Returns:The key password.
get_password()[source]

Returns the password for the key.

Return type:string
Returns:The key password.
get_type()[source]

Returns the type of the key, e.g. RSA or DSA.

Return type:string
Returns:The key type
keytypes = set([u'rsa', u'dss'])
set_filename(filename)[source]

Sets the name of the key file to use.

Parameters:filename (string) – The key filename.
set_password(password)[source]

Defines the password used for decrypting the key.

Parameters:password (string) – The key password.

Subpackages