GridCal.Engine.Replacements package

Submodules

GridCal.Engine.Replacements.mpiserve module

GridCal.Engine.Replacements.poap_controller module

Modified to fit GridCal’s purposes when using PySOT

class GridCal.Engine.Replacements.poap_controller.BaseWorkerThread(controller)

Bases: threading.Thread

Worker base class for use with the thread controller.

The BaseWorkerThread class has a run routine that actually handles the worker event loop, and a set of helper routines for dispatching messages into the worker event loop (usually from the controller) and dispatching messages to the controller (usually from the worker).

add_message(message)

Send message to be executed at the controller.

add_worker()

Add worker back to the work queue.

eval(record)

Start evaluation.

Args:
record: Function evaluation record
finish_cancelled(record)

Finish recording cancelled on a record and add ourselves back.

finish_killed(record)

Finish recording killed on a record and add ourselves back.

finish_success(record, value)

Finish successful work on a record and add ourselves back.

handle_eval(record)

Process an eval request.

handle_kill(record)

Process a kill request

handle_terminate()

Handle any cleanup on a terminate request

kill(record)

Send kill message to worker.

Args:
record: Function evaluation record
run()

Run requests as long as we get them.

terminate()

Send termination message to worker.

NB: if the worker is not running in a daemon thread, a call to the terminate method only returns after the the thread has terminated.

class GridCal.Engine.Replacements.poap_controller.BasicWorkerThread(controller, objective)

Bases: GridCal.Engine.Replacements.poap_controller.BaseWorkerThread

Basic worker for use with the thread controller.

The BasicWorkerThread calls a Python objective function when asked to do an evaluation. This is concurrent, but only results in parallelism if the objective function implementation itself allows parallelism (e.g. because it communicates with an external entity via a pipe, socket, or whatever).

handle_eval(record)

Process an eval request.

class GridCal.Engine.Replacements.poap_controller.Controller

Bases: object

Base class for controller.

Attributes:
strategy: Strategy for choosing optimization actions. fevals: Database of function evaluations. feval_callbacks: List of callbacks to execute on new eval record term_callbacks: List of callbacks to execute on termination
add_feval_callback(callback)

Add a callback for notification on new fevals.

add_term_callback(callback)

Add a callback for cleanup on termination.

add_timer(timeout, callback)

Add a task to be executed after a timeout (e.g. for monitoring).

Args:
timeout: Time to wait before execution callback: Function to call when timeout elapses
best_point(merit=None, filter=None)

Return the best point in the database satisfying some criterion.

Args:
merit: Function to minimize (default is r.value) filter: Predicate to use for filtering candidates
Returns:
Record minimizing merit() and satisfying filter(); or None if nothing satisfies the filter
call_term_callbacks()

Call termination callbacks.

can_work()

Return whether we can currently perform work.

new_feval(params, extra_args=None)

Add a function evaluation record to the database.

In addition to adding the record with status ‘pending’, we run the feval_callbacks on the new record.

Args:
params: Parameters to the objective function
Returns:
New EvalRecord object
ping()

Tell controller to consult strategies when possible (if asynchronous)

remove_feval_callback(callback)

Remove a callback from the feval callback list.

remove_term_callback(callback)

Remove a callback from the term callback list.

class GridCal.Engine.Replacements.poap_controller.Monitor(controller)

Bases: object

Monitor events observed by a controller.

The monitor object provides hooks to monitor the progress of an optimization run by a controller. Users should inherit from Monitor and add custom version of the methods

on_new_feval(self, record) on_update(self, record) on_complete(self, record) on_kill(self, record) on_cancel(self, record) on_terminate(self)
on_cancel(record)

Handle record cancelled

on_complete(record)

Handle feval completion

on_kill(record)

Handle record killed

on_new_feval(record)

Handle new function evaluation request.

on_terminate()

Handle termination.

on_update(record)

Handle feval update.

class GridCal.Engine.Replacements.poap_controller.ProcessWorkerThread(controller)

Bases: GridCal.Engine.Replacements.poap_controller.BaseWorkerThread

Subprocess worker for use with the thread controller.

The ProcessWorkerThread is meant for use as a base class. Implementations that inherit from ProcessWorkerThread should define a handle_eval method that sets the process field so that it can be interrupted if needed. This allows use of blocking communication primitives while at the same time allowing interruption.

kill(record)

Send kill message.

terminate()

Send termination message.

class GridCal.Engine.Replacements.poap_controller.ScriptedController

Bases: GridCal.Engine.Replacements.poap_controller.Controller

Run a test script of actions from the controller.

The ScriptedController is meant to test that a strategy adheres to an expected sequence of proposed actions in a given scenario.

Attributes:
strategy: Strategy for choosing optimization actions. fevals: Database of function evaluations
accept_eval(args=None, pred=None, skip=False)

Assert next proposal is an eval, which we accept.

Args:
args: expected evaluation args (if not None) pred: test predicate to run on proposal (if not None) skip: if True, skip over all None proposals
Returns:
proposal record
accept_kill(r=None, skip=False)

Assert next proposal is a kill, which we accept.

Args:
r: record to be killed. skip: if True, skip over all None proposals
accept_terminate(skip=False)

Assert next proposal is a kill, which we accept.

Args:
skip: if True, skip over all None proposals
add_timer(timeout, callback)

Add timer.

can_work()

Return True if worker available.

check_eval(proposal, args=None, pred=None)

Check whether a proposal is an expected eval proposal.

Args:
proposal: proposal to check args: expected evaluation args (if not None) pred: test predicate to run on proposal (if not None)
check_kill(proposal, r=None)

Check whether a proposal is an expected kill proposal.

Args:
proposal: proposal to check r: record to be killed (or None if no check)
check_terminate(proposal)

Check whether a proposal is an expected terminate proposal.

Args:
proposal: proposal to check
no_proposal()

Assert that next proposed action is None.

proposal(skip=False)

Return strategy proposal.

Args:
skip: if True, skip over all None proposals
reject_eval(args=None, pred=None, skip=False)

Assert next proposal is an eval, which we reject.

Args:
args: expected evaluation args (if not None) pred: test predicate to run on proposal (if not None) skip: if True, skip over all None proposals
reject_kill(r=None, skip=False)

Assert next proposal is a kill, which we reject.

Args:
r: record to be killed. skip: if True, skip over all None proposals
reject_terminate(skip=False)

Assert next proposal is a terminate, which we reject.

Args:
skip: if True, skip over all None proposals
set_worker(v)

Set worker availability status.

terminate()

Terminate the script.

class GridCal.Engine.Replacements.poap_controller.SerialController(objective, skip=False)

Bases: GridCal.Engine.Replacements.poap_controller.Controller

Serial optimization controller.

Attributes:
strategy: Strategy for choosing optimization actions. objective: Objective function fevals: Database of function evaluations skip: if True, skip over “None” proposals
run(merit=None, filter=None, reraise=True, stop_at=False, stop_value=0)

Run the optimization and return the best value.

Args:

merit: Function to minimize (default is r.value) filter: Predicate to use for filtering candidates reraise: Flag indicating whether exceptions in the

objective function evaluations should be re-raised, terminating the optimization.
Returns:
Record minimizing merit() and satisfying filter(); or None if nothing satisfies the filter
class GridCal.Engine.Replacements.poap_controller.SimTeamController(objective, delay, workers)

Bases: GridCal.Engine.Replacements.poap_controller.Controller

Simulated parallel optimization controller.

Run events in simulated time. If two events are scheduled at the same time, we prioritize by when the event was added to the queue.

Attributes:
strategy: Strategy for choosing optimization actions. objective: Objective function delay: Time delay function workers: Number of workers available fevals: Database of function evaluations time: Current simulated time time_events: Time-stamped event heap
add_timer(timeout, event)

Add a task to be executed after a timeout (e.g. for monitoring).

Args:
timeout: Time to wait before execution callback: Function to call when timeout elapses
advance_time()

Advance time to the next event.

can_work()

Check if there are workers available.

kill_work(proposal)

Submit a kill event.

run(merit=None, filter=None)

Run the optimization and return the best value.

Args:
merit: Function to minimize (default is r.value) filter: Predicate to use for filtering candidates
Returns:
Record minimizing merit() and satisfying filter(); or None if nothing satisfies the filter
submit_work(proposal)

Submit a work event.

class GridCal.Engine.Replacements.poap_controller.ThreadController

Bases: GridCal.Engine.Replacements.poap_controller.Controller

Thread-based optimization controller.

The optimizer dispatches work to a queue of workers. Each worker has methods of the form

worker.eval(record) worker.kill(record)

These methods are asynchronous: they start a function evaluation or termination, but do not necessarily complete it. The worker must respond to eval requests, but may ignore kill requests. On eval requests, the worker should either attempt the evaluation or mark the record as killed. The worker sends status updates back to the controller in terms of lambdas (executed at the controller) that update the relevant record. When the worker becomes available again, it should use add_worker to add itself back to the queue.

Attributes:
strategy: Strategy for choosing optimization actions. fevals: Database of function evaluations workers: Queue of available worker threads messages: Queue of messages from workers
add_message(message=None)

Queue up a message.

Args:
message: callback function with no arguments or None (default)
if None, a dummy message is queued to ping the controller
add_timer(timeout, callback)

Add a task to be executed after a timeout (e.g. for monitoring).

Args:
timeout: Time to wait before execution callback: Function to call when timeout elapses
add_worker(worker)

Add a worker and queue a ‘wake-up’ message.

Args:
worker: a worker thread object
can_work()

Claim we can work if a worker is available.

launch_worker(worker, daemon=False)

Launch and take ownership of a new worker thread.

Args:

worker: a worker thread object daemon: if True, the worker is launched in a daemon thread

(default is False)
ping()

Tell controller to consult strategies when possible

run(merit=None, filter=None)

Run the optimization and return the best value.

Args:
merit: Function to minimize (default is r.value) filter: Predicate to use for filtering candidates
Returns:
Record minimizing merit() and satisfying filter(); or None if nothing satisfies the filter

GridCal.Engine.Replacements.strategy module

class GridCal.Engine.Replacements.strategy.AddArgStrategy(strategy, extra_args=None)

Bases: GridCal.Engine.Replacements.strategy.BaseStrategy

Add extra arguments to evaluation proposals.

Decorates evaluation proposals with extra arguments. These may reflect termination criteria, advice to workers about which of several methods to use, etc. The extra arguments can be static (via the extra_args attribute), or they may be returned from the get_extra_args function, which child classes can override. The strategy overwrites any existing list of extra arguments, so it is probably a mistake to compose strategies in a way that involves multiple objects of this type.

Attributes:
strategy: Parent strategy extra_args: Extra arguments to be added
add_extra_args(record)

Add any extra arguments to an eval record.

Args:
record: Record to be decorated
on_reply_accept(proposal)

Handle proposal acceptance.

propose_action()
class GridCal.Engine.Replacements.strategy.BaseStrategy

Bases: object

Base strategy class.

The BaseStrategy class provides some support for the basic callback flow common to most strategies: handlers are called when an evaluation is accepted or rejected; and if an evaluation proposal is accepted, the record is decorated so that an on_update handler is called for subsequent updates.

Note that not all strategies follow this pattern – the only requirement for a strategy is that it implement propose_action.

decorate_proposal(proposal)

Add a callback to an existing proposal (for nested strategies).

on_complete(record)

Process completed record.

on_kill(record)

Process killed or cancelled record.

on_kill_reply(proposal)

Default handling of kill proposal.

on_kill_reply_accept(proposal)

Handle proposal acceptance.

on_kill_reply_reject(proposal)

Handle proposal rejection.

on_reply(proposal)

Default handling of eval proposal.

on_reply_accept(proposal)

Handle proposal acceptance.

on_reply_reject(proposal)

Handle proposal rejection.

on_terminate_reply(proposal)

Default handling of terminate proposal.

on_terminate_reply_accept(proposal)

Handle proposal acceptance.

on_terminate_reply_reject(proposal)

Handle proposal rejection.

on_update(record)

Process update.

propose_eval(*args)

Generate an eval proposal with a callback to on_reply.

propose_kill(r)

Generate a kill proposal with a callback to on_reply_kill.

propose_terminate()

Generate a terminate proposal with a callback to on_terminate.

class GridCal.Engine.Replacements.strategy.ChaosMonkeyStrategy(controller, strategy, mtbf=1)

Bases: object

Randomly kill running function evaluations.

The ChaosMonkeyStrategy kills function evaluations at random from among all active function evaluations. Attacks are associated with a Poisson process where the mean time between failures is specified at startup. Useful mostly for testing the resilience of strategies to premature termination of their evaluations. The controller may decide to ignore the proposed kill actions, in which case the monkey’s attacks are ultimately futile.

on_new_feval(record)

On every feval, add a callback.

on_timer()
on_update(record)

On every completion, remove from list

propose_action()
class GridCal.Engine.Replacements.strategy.CheckWorkerStrategy(controller, strategy)

Bases: object

Preemptively kill eval proposals when there are no workers.

A strategy like the fixed sampler is simple-minded, and will propose a function evaluation even if the controller has no workers available to carry it out. This wrapper strategy intercepts proposals from a wrapped strategy like the fixed sampler, and only submits evaluation proposals if workers are available.

propose_action()

Generate filtered action proposal.

class GridCal.Engine.Replacements.strategy.CoroutineBatchStrategy(coroutine, rvalue=<function CoroutineBatchStrategy.<lambda>>)

Bases: GridCal.Engine.Replacements.strategy.BaseStrategy

Event-driven to synchronous parallel adapter using coroutines.

The coroutine strategy runs a synchronous parallel optimization algorithm in a Python coroutine, with which the strategy communicates via send/yield directives. The optimization coroutine yields batches of parameters (as lists) at which it would like function values; the strategy then requests the function evaluation and returns the records associated with those function evaluations when all have been completed.

NB: Within a given batch, function evaluations are attempted in the reverse of the order in which they are specified. This should make no difference for most things (the batch is not assumed to have any specific order), but it is significant for testing.

Attributes:
coroutine: Optimizer coroutine retry: Retry strategy for in-flight actions results: Completion records to be returned to the coroutine
on_complete(record)

Return or re-request on completion or cancellation.

on_reply_accept(proposal)

If proposal accepted, wait for result.

propose_action()

If we have a pending request, propose it.

start_batch(xs)

Start evaluation of a batch of points.

class GridCal.Engine.Replacements.strategy.CoroutineStrategy(coroutine, rvalue=<function CoroutineStrategy.<lambda>>)

Bases: GridCal.Engine.Replacements.strategy.BaseStrategy

Event-driven to serial adapter using coroutines.

The coroutine strategy runs a standard sequential optimization algorithm in a Python coroutine, with which the strategy communicates via send/yield directives. The optimization coroutine yields the parameters at which it would like function values; the strategy then requests the function evaluation and returns the value with a send command when it becomes available.

Attributes:
coroutine: Optimizer coroutine rvalue: Function to map records to values retry: Retry manager
on_complete(record)

Return point to coroutine

propose_action()

If we have a pending request, propose it.

class GridCal.Engine.Replacements.strategy.EvalRecord(params, extra_args=None, status='pending')

Bases: object

Represent progress of a function evaluation.

An evaluation record includes the evaluation point, status of the evaluation, and a list of callbacks. The status may be pending, running, killed (deliberately), cancelled (due to a crash or failure) or completed. Once the function evaluation is completed, the value is stored as an attribute in the evaluation record. Other information, such as gradients, Hessians, or function values used to compute constraints, may also be stored in the record.

The evaluation record callbacks are triggered on any relevant event, including not only status changes but also intermediate results. The nature of these intermediate results (lower bounds, initial estimates, etc) is somewhat application-dependent.

NB: Callbacks that handle record updates should be insensitive to order: it is important not to modify attributes that might be used by later callbacks (e.g. the proposal.accepted flag).

Attributes:
params: Evaluation point for the function status: Status of the evaluation (pending, running, killed, completed) value: Return value (if completed) callbacks: Functions to call on status updates
add_callback(callback)

Add a callback for update events.

cancel()

Change status to ‘cancelled’ and execute callbacks.

complete(value)

Change status to ‘completed’ and execute callbacks.

is_cancelled

Check whether evaluation was cancelled

is_completed

Check for successful completion of evaluation

is_done

Check whether the status indicates the evaluation is finished.

is_killed

Check whether evaluation is killed

is_pending

Check if status is pending.

is_running

Check if status is running.

kill()

Change status to ‘killed’ and execute callbacks.

remove_callback(callback)

Remove a callback subscriber.

running()

Change status to ‘running’ and execute callbacks.

status
update(**kwargs)

Update fields and execute callbacks.

update_dict(dict)

Update fields and execute callbacks.

Args:
dict: Dictionary of attributes to set on the record
class GridCal.Engine.Replacements.strategy.FixedSampleStrategy(points)

Bases: GridCal.Engine.Replacements.strategy.BaseStrategy

Sample at a fixed set of points.

The fixed sampling strategy is appropriate for any non-adaptive sampling scheme. Since the strategy is non-adaptive, we can employ as many available workers as we have points to process. We keep trying any evaluation that fails, and suggest termination only when all evaluations are complete. The points in the experimental design can be provided as any iterable object (e.g. a list or a generator function). One can use a generator for an infinite sequence if the fixed sampling strategy is used in combination with a strategy that provides a termination criterion.

propose_action()

Propose an action based on outstanding points.

class GridCal.Engine.Replacements.strategy.InputStrategy(controller, strategy)

Bases: GridCal.Engine.Replacements.strategy.BaseStrategy

Insert requests from the outside world (e.g. from a GUI).

eval(params, retry=True)

Request a new function evaluation

kill(record, retry=False)

Request a function evaluation be killed

propose_action()
terminate(retry=True)

Request termination of the optimization

class GridCal.Engine.Replacements.strategy.MaxEvalStrategy(controller, max_counter)

Bases: object

Recommend termination of the iteration after some number of evals.

Recommends termination after observing some number of completed function evaluations. We allow more than the requisite number of evaluations to be started, since there’s always the possibility that one of our outstanding evaluations won’t finish.

Attributes:
counter: Number of completed evals max_counter: Max number of evals
on_new_feval(record)

On every feval, add a callback.

on_update(record)

On every completion, increment the counter.

propose_action()

Propose termination once the eval counter is high enough.

class GridCal.Engine.Replacements.strategy.MultiStartStrategy(controller, strategies)

Bases: object

Merge several strategies by taking the first valid eval proposal.

This strategy is similar to the SimpleMergedStrategy, except that we only terminate once all the worker strategies have voted to terminate.

Attributes:
controller: Controller object used to determine whether we can eval strategies: Prioritized list of strategies
propose_action()

Go through strategies in order and choose the first valid action. Terminate iff all workers vote to terminate.

class GridCal.Engine.Replacements.strategy.PromiseStrategy(rvalue=<function PromiseStrategy.<lambda>>, block=True)

Bases: GridCal.Engine.Replacements.strategy.BaseStrategy

Provides a promise-based asynchronous evaluation interface.

A promise (aka a future) is a common concurrent programming abstraction. A caller requests an asynchronous evaluation, and the call returns immediately with a promise object. The caller can check the promise object to see if it has a value ready, or wait for the value to become ready. The callee can set the value when it is ready.

Attributes:
proposalq: Queue of proposed actions caused by optimizer. valueq: Queue of function values from proposed actions. rvalue: Function to extract return value from a record block: Flag whether to block on proposal pop
class Promise(valueq)

Bases: object

Evaluation promise.

Properties:
value: Promised value. Block on read if not ready.
ready()

Check whether the value is ready (at consumer)

value

Wait on the value (at consumer)

blocking_eval(*args)

Request a function evaluation and block until done.

blocking_evals(xs)

Request a list of function evaluations.

on_complete(record)

Send the value to the consumer via the promise.

on_kill(record)

Re-submit the proposal with the same promise.

on_reply_accept(proposal)

Make sure we copy over the promise to the feval record.

on_reply_reject(proposal)

Re-submit the proposal with the same promise.

on_terminate()

Throw an exception at the consumer if still running on termination

on_terminate_reply_reject(proposal)

Re-submit the termination proposal.

promise_eval(*args)

Request a function evaluation and return a promise object.

propose_action()

Provide proposals from the queue.

terminate()

Request termination.

class GridCal.Engine.Replacements.strategy.Proposal(action, *args)

Bases: object

Represent a proposed action.

We currently recognize three types of proposals: evaluation requests (“eval”), requests to stop an evaluation (“kill”), and requests to terminate the optimization (“terminate”). When an evaluation proposal is accepted, the controller adds an evaluation record to the proposal before notifying subscribers.

NB: Callbacks that handle proposal accept/reject should be insensitive to order: it is important not to modify attributes that might be used by later callbacks (e.g. the proposal.accepted flag).

Attributes:
action: String describing the action args: Tuple of arguments to pass to the action accepted: Flag set on accept/reject decision callbacks: Functions to call on accept/reject of action record: Evaluation record for accepted evaluation proposals
accept()

Mark the action as accepted and execute callbacks.

add_callback(callback)

Add a callback subscriber to the action.

copy()

Make a copy of the proposal.

reject()

Mark action as rejected and execute callbacks.

remove_callback(callback)

Remove a callback subscriber.

class GridCal.Engine.Replacements.strategy.RetryStrategy

Bases: GridCal.Engine.Replacements.strategy.BaseStrategy

Retry strategy class.

The RetryStrategy class manages a queue of proposals to be retried, either because they were rejected or because they correspond to a function evaluation that was killed.

If a proposal is submitted to a retry strategy and it has already been marked with the .retry attribute, we do not attempt to manage retries, as another object is assumed to have taken responsibility for retrying the proposal on failure. This prevents us from having redundant retries.

When a proposal is submitted to the retry class, we make a copy (the proposal draft) for safe-keeping. If at any point it is necessary to resubmit, we submit the draft copy to the retry class. The draft will have any modifications or callbacks that were added before the proposal reached this strategy, and none of those that were added afterward (including those added afterward by the RetryStrategy itself). The intent is that retried proposals inserted into the strategy hierarchy at the place where they were recorded should require no special treatment to avoid adding multiple copies of callbacks (for example).

Attributes:
proposals: Queue of outstanding proposals num_eval_pending: Number of pending evaluations num_eval_running: Number of running evaluations num_eval_outstanding: Number of outstanding evaluations
empty()

Check if the queue is empty

get()

Pop a proposal from the queue.

num_eval_outstanding

Number of outstanding function evaluations.

on_complete(record)

Clean up after completed eval+retry

on_kill(record)

Resubmit proposal for killed or cancelled eval+retry

on_kill_reply_reject(proposal)

Resubmit rejected kill+retry proposal

on_reply_accept(proposal)

Process accepted eval+retry proposal

on_reply_reject(proposal)

Resubmit rejected eval+retry proposal

on_terminate_reply_reject(proposal)

Resubmit rejected termination+retry proposal

put(proposal)

Put a non-retry proposal in the queue.

rput(proposal)

Put a retry proposal in the queue.

exception GridCal.Engine.Replacements.strategy.RunTerminatedException

Bases: Exception

class GridCal.Engine.Replacements.strategy.SimpleMergedStrategy(controller, strategies)

Bases: object

Merge several strategies by taking the first valid proposal from them.

The simplest combination strategy is to keep a list of several possible strategies in some priority order, query them for proposals in priority order, and pass on the first plausible proposal to the controller.

Attributes:
controller: Controller object used to determine whether we can eval strategies: Prioritized list of strategies
propose_action()

Go through strategies in order and choose the first valid action.

class GridCal.Engine.Replacements.strategy.ThreadStrategy(controller, optimizer, rvalue=<function ThreadStrategy.<lambda>>)

Bases: GridCal.Engine.Replacements.strategy.PromiseStrategy

Event-driven to serial adapter using threads.

The thread strategy runs a standard sequential optimization algorithm in a separate thread of control, with which the strategy communicates via promises. The optimizer thread intercepts function evaluation requests and completion and turns them into proposals for the strategy, which it places in a proposal queue. The optimizer then waits for the requested values (if any) to appear in the reply queue.

Attributes:
optimizer: Optimizer function (takes objective as an argument) proposalq: Queue of proposed actions caused by optimizer. valueq: Queue of function values from proposed actions. thread: Thread in which the optimizer runs. proposal: Proposal that the strategy is currently requesting. rvalue: Function mapping records to values
class OptimizerThread(strategy, optimizer)

Bases: threading.Thread

run()

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

on_terminate()

Throw an exception at the consumer if still running on termination

GridCal.Engine.Replacements.tcpserve module

class GridCal.Engine.Replacements.tcpserve.ProcessSocketWorker(sockname, retries=0)

Bases: GridCal.Engine.Replacements.tcpserve.SocketWorker

Socket worker that runs an evaluation in a subprocess

The ProcessSocketWorker is a base class for simulations that run a simulation in an external subprocess. This class provides functionality just to allow graceful termination of the external simulations.

Attributes:
process: Handle for external subprocess
kill(record_id)

Kill a function evaluation

kill_process()

Kill the child process

terminate()

Terminate the worker

class GridCal.Engine.Replacements.tcpserve.SimpleSocketWorker(objective, sockname, retries=0)

Bases: GridCal.Engine.Replacements.tcpserve.SocketWorker

Simple socket worker that runs a local objective function

The SimpleSocketWorker is a socket worker that runs a local Python function and returns the result. It is probably mostly useful for testing – the ProcessSocketWorker is a better option for external simulations.

eval(record_id, params)

Evaluate the function and send back a result.

If the function evaluation crashes, we send back a ‘cancel’ request for the record. If, on the other hand, there is a problem with calling send, we probably want to let the worker error out.

Args:
record_id: Feval record identifier used by server/controller params: Parameters sent to the function to be evaluated
class GridCal.Engine.Replacements.tcpserve.SocketWorker(sockname, retries=0)

Bases: object

Base class for workers that connect to SocketServer interface

The socket server interface is a server to which workers can connect. The socket worker is the client to that interface. It connects to a given TCP/IP port, then attempts to do work on the controller’s behalf.

Attributes:
running: True if the socket is active sock: Worker TCP socket
eval(record_id, params)

Compute a function value

kill(record_id)

Kill a function evaluation

marshall(*args)

Marshall data to wire format

run(loop=True)

Main loop

send(*args)

Send a message to the controller

terminate()

Terminate the worker

unmarshall(data)

Convert data from wire format back to Python tuple

class GridCal.Engine.Replacements.tcpserve.SocketWorkerHandler(request, client_address, server)

Bases: socketserver.BaseRequestHandler

Manage a remote worker for a thread controller.

The SocketWorkerHandler is a request handler for incoming workers. It implements the socketserver request handler interface, and also the worker interface expected by the ThreadController.

eval(record)

Send an evaluation request to remote worker

handle()

Main event loop called from SocketServer

kill(record)

Send a kill request to a remote worker

terminate()

Send a termination request to a remote worker

class GridCal.Engine.Replacements.tcpserve.ThreadedTCPServer(sockname=('localhost', 0), strategy=None, handlers={})

Bases: socketserver.ThreadingMixIn, socketserver.TCPServer, object

SocketServer interface for workers to connect to controller.

The socket server interface lets workers connect to a given TCP/IP port and exchange updates with the controller.

The server sends messages of the form

(‘eval’, record_id, args, extra_args) (‘eval’, record_id, args) (‘kill’, record_id) (‘terminate’)

The default messages received are

(‘running’, record_id) (‘kill’, record_id) (‘cancel’, record_id) (‘complete’, record_id, value)

The set of handlers can also be extended with a dictionary of named callbacks to be invoked whenever a record update comes in. For example, to set a lower bound field, we might use the handler

def set_lb(rec, value):
rec.lb = value

handlers = {‘lb’ : set_lb }

This is useful for adding new types of updates without mucking around in the EvalRecord implementation.

Attributes:
controller: ThreadController that manages the optimization handlers: dictionary of specialized message handlers strategy: redirects to the controller strategy
marshall(*args)

Convert an argument list to wire format.

run(merit=<function ThreadedTCPServer.<lambda>>, filter=None)
sockname
strategy
unmarshall(data)

Convert wire format back to Python arg list.