GridCal.Engine.Replacements package¶
Submodules¶
GridCal.Engine.Replacements.mpiserve module¶
GridCal.Engine.Replacements.poap_controller module¶
Modified to fit GridCal’s purposes when using PySOT
-
class
GridCal.Engine.Replacements.poap_controller.
BaseWorkerThread
(controller)¶ Bases:
threading.Thread
Worker base class for use with the thread controller.
The BaseWorkerThread class has a run routine that actually handles the worker event loop, and a set of helper routines for dispatching messages into the worker event loop (usually from the controller) and dispatching messages to the controller (usually from the worker).
-
add_message
(message)¶ Send message to be executed at the controller.
-
add_worker
()¶ Add worker back to the work queue.
-
eval
(record)¶ Start evaluation.
- Args:
- record: Function evaluation record
-
finish_cancelled
(record)¶ Finish recording cancelled on a record and add ourselves back.
-
finish_killed
(record)¶ Finish recording killed on a record and add ourselves back.
-
finish_success
(record, value)¶ Finish successful work on a record and add ourselves back.
-
handle_eval
(record)¶ Process an eval request.
-
handle_kill
(record)¶ Process a kill request
-
handle_terminate
()¶ Handle any cleanup on a terminate request
-
kill
(record)¶ Send kill message to worker.
- Args:
- record: Function evaluation record
-
run
()¶ Run requests as long as we get them.
-
terminate
()¶ Send termination message to worker.
NB: if the worker is not running in a daemon thread, a call to the terminate method only returns after the the thread has terminated.
-
-
class
GridCal.Engine.Replacements.poap_controller.
BasicWorkerThread
(controller, objective)¶ Bases:
GridCal.Engine.Replacements.poap_controller.BaseWorkerThread
Basic worker for use with the thread controller.
The BasicWorkerThread calls a Python objective function when asked to do an evaluation. This is concurrent, but only results in parallelism if the objective function implementation itself allows parallelism (e.g. because it communicates with an external entity via a pipe, socket, or whatever).
-
handle_eval
(record)¶ Process an eval request.
-
-
class
GridCal.Engine.Replacements.poap_controller.
Controller
¶ Bases:
object
Base class for controller.
- Attributes:
- strategy: Strategy for choosing optimization actions. fevals: Database of function evaluations. feval_callbacks: List of callbacks to execute on new eval record term_callbacks: List of callbacks to execute on termination
-
add_feval_callback
(callback)¶ Add a callback for notification on new fevals.
-
add_term_callback
(callback)¶ Add a callback for cleanup on termination.
-
add_timer
(timeout, callback)¶ Add a task to be executed after a timeout (e.g. for monitoring).
- Args:
- timeout: Time to wait before execution callback: Function to call when timeout elapses
-
best_point
(merit=None, filter=None)¶ Return the best point in the database satisfying some criterion.
- Args:
- merit: Function to minimize (default is r.value) filter: Predicate to use for filtering candidates
- Returns:
- Record minimizing merit() and satisfying filter(); or None if nothing satisfies the filter
-
call_term_callbacks
()¶ Call termination callbacks.
-
can_work
()¶ Return whether we can currently perform work.
-
new_feval
(params, extra_args=None)¶ Add a function evaluation record to the database.
In addition to adding the record with status ‘pending’, we run the feval_callbacks on the new record.
- Args:
- params: Parameters to the objective function
- Returns:
- New EvalRecord object
-
ping
()¶ Tell controller to consult strategies when possible (if asynchronous)
-
remove_feval_callback
(callback)¶ Remove a callback from the feval callback list.
-
remove_term_callback
(callback)¶ Remove a callback from the term callback list.
-
class
GridCal.Engine.Replacements.poap_controller.
Monitor
(controller)¶ Bases:
object
Monitor events observed by a controller.
The monitor object provides hooks to monitor the progress of an optimization run by a controller. Users should inherit from Monitor and add custom version of the methods
on_new_feval(self, record) on_update(self, record) on_complete(self, record) on_kill(self, record) on_cancel(self, record) on_terminate(self)-
on_cancel
(record)¶ Handle record cancelled
-
on_complete
(record)¶ Handle feval completion
-
on_kill
(record)¶ Handle record killed
-
on_new_feval
(record)¶ Handle new function evaluation request.
-
on_terminate
()¶ Handle termination.
-
on_update
(record)¶ Handle feval update.
-
-
class
GridCal.Engine.Replacements.poap_controller.
ProcessWorkerThread
(controller)¶ Bases:
GridCal.Engine.Replacements.poap_controller.BaseWorkerThread
Subprocess worker for use with the thread controller.
The ProcessWorkerThread is meant for use as a base class. Implementations that inherit from ProcessWorkerThread should define a handle_eval method that sets the process field so that it can be interrupted if needed. This allows use of blocking communication primitives while at the same time allowing interruption.
-
kill
(record)¶ Send kill message.
-
terminate
()¶ Send termination message.
-
-
class
GridCal.Engine.Replacements.poap_controller.
ScriptedController
¶ Bases:
GridCal.Engine.Replacements.poap_controller.Controller
Run a test script of actions from the controller.
The ScriptedController is meant to test that a strategy adheres to an expected sequence of proposed actions in a given scenario.
- Attributes:
- strategy: Strategy for choosing optimization actions. fevals: Database of function evaluations
-
accept_eval
(args=None, pred=None, skip=False)¶ Assert next proposal is an eval, which we accept.
- Args:
- args: expected evaluation args (if not None) pred: test predicate to run on proposal (if not None) skip: if True, skip over all None proposals
- Returns:
- proposal record
-
accept_kill
(r=None, skip=False)¶ Assert next proposal is a kill, which we accept.
- Args:
- r: record to be killed. skip: if True, skip over all None proposals
-
accept_terminate
(skip=False)¶ Assert next proposal is a kill, which we accept.
- Args:
- skip: if True, skip over all None proposals
-
add_timer
(timeout, callback)¶ Add timer.
-
can_work
()¶ Return True if worker available.
-
check_eval
(proposal, args=None, pred=None)¶ Check whether a proposal is an expected eval proposal.
- Args:
- proposal: proposal to check args: expected evaluation args (if not None) pred: test predicate to run on proposal (if not None)
-
check_kill
(proposal, r=None)¶ Check whether a proposal is an expected kill proposal.
- Args:
- proposal: proposal to check r: record to be killed (or None if no check)
-
check_terminate
(proposal)¶ Check whether a proposal is an expected terminate proposal.
- Args:
- proposal: proposal to check
-
no_proposal
()¶ Assert that next proposed action is None.
-
proposal
(skip=False)¶ Return strategy proposal.
- Args:
- skip: if True, skip over all None proposals
-
reject_eval
(args=None, pred=None, skip=False)¶ Assert next proposal is an eval, which we reject.
- Args:
- args: expected evaluation args (if not None) pred: test predicate to run on proposal (if not None) skip: if True, skip over all None proposals
-
reject_kill
(r=None, skip=False)¶ Assert next proposal is a kill, which we reject.
- Args:
- r: record to be killed. skip: if True, skip over all None proposals
-
reject_terminate
(skip=False)¶ Assert next proposal is a terminate, which we reject.
- Args:
- skip: if True, skip over all None proposals
-
set_worker
(v)¶ Set worker availability status.
-
terminate
()¶ Terminate the script.
-
class
GridCal.Engine.Replacements.poap_controller.
SerialController
(objective, skip=False)¶ Bases:
GridCal.Engine.Replacements.poap_controller.Controller
Serial optimization controller.
- Attributes:
- strategy: Strategy for choosing optimization actions. objective: Objective function fevals: Database of function evaluations skip: if True, skip over “None” proposals
-
run
(merit=None, filter=None, reraise=True, stop_at=False, stop_value=0)¶ Run the optimization and return the best value.
- Args:
merit: Function to minimize (default is r.value) filter: Predicate to use for filtering candidates reraise: Flag indicating whether exceptions in the
objective function evaluations should be re-raised, terminating the optimization.- Returns:
- Record minimizing merit() and satisfying filter(); or None if nothing satisfies the filter
-
class
GridCal.Engine.Replacements.poap_controller.
SimTeamController
(objective, delay, workers)¶ Bases:
GridCal.Engine.Replacements.poap_controller.Controller
Simulated parallel optimization controller.
Run events in simulated time. If two events are scheduled at the same time, we prioritize by when the event was added to the queue.
- Attributes:
- strategy: Strategy for choosing optimization actions. objective: Objective function delay: Time delay function workers: Number of workers available fevals: Database of function evaluations time: Current simulated time time_events: Time-stamped event heap
-
add_timer
(timeout, event)¶ Add a task to be executed after a timeout (e.g. for monitoring).
- Args:
- timeout: Time to wait before execution callback: Function to call when timeout elapses
-
advance_time
()¶ Advance time to the next event.
-
can_work
()¶ Check if there are workers available.
-
kill_work
(proposal)¶ Submit a kill event.
-
run
(merit=None, filter=None)¶ Run the optimization and return the best value.
- Args:
- merit: Function to minimize (default is r.value) filter: Predicate to use for filtering candidates
- Returns:
- Record minimizing merit() and satisfying filter(); or None if nothing satisfies the filter
-
submit_work
(proposal)¶ Submit a work event.
-
class
GridCal.Engine.Replacements.poap_controller.
ThreadController
¶ Bases:
GridCal.Engine.Replacements.poap_controller.Controller
Thread-based optimization controller.
The optimizer dispatches work to a queue of workers. Each worker has methods of the form
worker.eval(record) worker.kill(record)These methods are asynchronous: they start a function evaluation or termination, but do not necessarily complete it. The worker must respond to eval requests, but may ignore kill requests. On eval requests, the worker should either attempt the evaluation or mark the record as killed. The worker sends status updates back to the controller in terms of lambdas (executed at the controller) that update the relevant record. When the worker becomes available again, it should use add_worker to add itself back to the queue.
- Attributes:
- strategy: Strategy for choosing optimization actions. fevals: Database of function evaluations workers: Queue of available worker threads messages: Queue of messages from workers
-
add_message
(message=None)¶ Queue up a message.
- Args:
- message: callback function with no arguments or None (default)
- if None, a dummy message is queued to ping the controller
-
add_timer
(timeout, callback)¶ Add a task to be executed after a timeout (e.g. for monitoring).
- Args:
- timeout: Time to wait before execution callback: Function to call when timeout elapses
-
add_worker
(worker)¶ Add a worker and queue a ‘wake-up’ message.
- Args:
- worker: a worker thread object
-
can_work
()¶ Claim we can work if a worker is available.
-
launch_worker
(worker, daemon=False)¶ Launch and take ownership of a new worker thread.
- Args:
worker: a worker thread object daemon: if True, the worker is launched in a daemon thread
(default is False)
-
ping
()¶ Tell controller to consult strategies when possible
-
run
(merit=None, filter=None)¶ Run the optimization and return the best value.
- Args:
- merit: Function to minimize (default is r.value) filter: Predicate to use for filtering candidates
- Returns:
- Record minimizing merit() and satisfying filter(); or None if nothing satisfies the filter
GridCal.Engine.Replacements.strategy module¶
-
class
GridCal.Engine.Replacements.strategy.
AddArgStrategy
(strategy, extra_args=None)¶ Bases:
GridCal.Engine.Replacements.strategy.BaseStrategy
Add extra arguments to evaluation proposals.
Decorates evaluation proposals with extra arguments. These may reflect termination criteria, advice to workers about which of several methods to use, etc. The extra arguments can be static (via the extra_args attribute), or they may be returned from the get_extra_args function, which child classes can override. The strategy overwrites any existing list of extra arguments, so it is probably a mistake to compose strategies in a way that involves multiple objects of this type.
- Attributes:
- strategy: Parent strategy extra_args: Extra arguments to be added
-
add_extra_args
(record)¶ Add any extra arguments to an eval record.
- Args:
- record: Record to be decorated
-
on_reply_accept
(proposal)¶ Handle proposal acceptance.
-
propose_action
()¶
-
class
GridCal.Engine.Replacements.strategy.
BaseStrategy
¶ Bases:
object
Base strategy class.
The BaseStrategy class provides some support for the basic callback flow common to most strategies: handlers are called when an evaluation is accepted or rejected; and if an evaluation proposal is accepted, the record is decorated so that an on_update handler is called for subsequent updates.
Note that not all strategies follow this pattern – the only requirement for a strategy is that it implement propose_action.
-
decorate_proposal
(proposal)¶ Add a callback to an existing proposal (for nested strategies).
-
on_complete
(record)¶ Process completed record.
-
on_kill
(record)¶ Process killed or cancelled record.
-
on_kill_reply
(proposal)¶ Default handling of kill proposal.
-
on_kill_reply_accept
(proposal)¶ Handle proposal acceptance.
-
on_kill_reply_reject
(proposal)¶ Handle proposal rejection.
-
on_reply
(proposal)¶ Default handling of eval proposal.
-
on_reply_accept
(proposal)¶ Handle proposal acceptance.
-
on_reply_reject
(proposal)¶ Handle proposal rejection.
-
on_terminate_reply
(proposal)¶ Default handling of terminate proposal.
-
on_terminate_reply_accept
(proposal)¶ Handle proposal acceptance.
-
on_terminate_reply_reject
(proposal)¶ Handle proposal rejection.
-
on_update
(record)¶ Process update.
-
propose_eval
(*args)¶ Generate an eval proposal with a callback to on_reply.
-
propose_kill
(r)¶ Generate a kill proposal with a callback to on_reply_kill.
-
propose_terminate
()¶ Generate a terminate proposal with a callback to on_terminate.
-
-
class
GridCal.Engine.Replacements.strategy.
ChaosMonkeyStrategy
(controller, strategy, mtbf=1)¶ Bases:
object
Randomly kill running function evaluations.
The ChaosMonkeyStrategy kills function evaluations at random from among all active function evaluations. Attacks are associated with a Poisson process where the mean time between failures is specified at startup. Useful mostly for testing the resilience of strategies to premature termination of their evaluations. The controller may decide to ignore the proposed kill actions, in which case the monkey’s attacks are ultimately futile.
-
on_new_feval
(record)¶ On every feval, add a callback.
-
on_timer
()¶
-
on_update
(record)¶ On every completion, remove from list
-
propose_action
()¶
-
-
class
GridCal.Engine.Replacements.strategy.
CheckWorkerStrategy
(controller, strategy)¶ Bases:
object
Preemptively kill eval proposals when there are no workers.
A strategy like the fixed sampler is simple-minded, and will propose a function evaluation even if the controller has no workers available to carry it out. This wrapper strategy intercepts proposals from a wrapped strategy like the fixed sampler, and only submits evaluation proposals if workers are available.
-
propose_action
()¶ Generate filtered action proposal.
-
-
class
GridCal.Engine.Replacements.strategy.
CoroutineBatchStrategy
(coroutine, rvalue=<function CoroutineBatchStrategy.<lambda>>)¶ Bases:
GridCal.Engine.Replacements.strategy.BaseStrategy
Event-driven to synchronous parallel adapter using coroutines.
The coroutine strategy runs a synchronous parallel optimization algorithm in a Python coroutine, with which the strategy communicates via send/yield directives. The optimization coroutine yields batches of parameters (as lists) at which it would like function values; the strategy then requests the function evaluation and returns the records associated with those function evaluations when all have been completed.
NB: Within a given batch, function evaluations are attempted in the reverse of the order in which they are specified. This should make no difference for most things (the batch is not assumed to have any specific order), but it is significant for testing.
- Attributes:
- coroutine: Optimizer coroutine retry: Retry strategy for in-flight actions results: Completion records to be returned to the coroutine
-
on_complete
(record)¶ Return or re-request on completion or cancellation.
-
on_reply_accept
(proposal)¶ If proposal accepted, wait for result.
-
propose_action
()¶ If we have a pending request, propose it.
-
start_batch
(xs)¶ Start evaluation of a batch of points.
-
class
GridCal.Engine.Replacements.strategy.
CoroutineStrategy
(coroutine, rvalue=<function CoroutineStrategy.<lambda>>)¶ Bases:
GridCal.Engine.Replacements.strategy.BaseStrategy
Event-driven to serial adapter using coroutines.
The coroutine strategy runs a standard sequential optimization algorithm in a Python coroutine, with which the strategy communicates via send/yield directives. The optimization coroutine yields the parameters at which it would like function values; the strategy then requests the function evaluation and returns the value with a send command when it becomes available.
- Attributes:
- coroutine: Optimizer coroutine rvalue: Function to map records to values retry: Retry manager
-
on_complete
(record)¶ Return point to coroutine
-
propose_action
()¶ If we have a pending request, propose it.
-
class
GridCal.Engine.Replacements.strategy.
EvalRecord
(params, extra_args=None, status='pending')¶ Bases:
object
Represent progress of a function evaluation.
An evaluation record includes the evaluation point, status of the evaluation, and a list of callbacks. The status may be pending, running, killed (deliberately), cancelled (due to a crash or failure) or completed. Once the function evaluation is completed, the value is stored as an attribute in the evaluation record. Other information, such as gradients, Hessians, or function values used to compute constraints, may also be stored in the record.
The evaluation record callbacks are triggered on any relevant event, including not only status changes but also intermediate results. The nature of these intermediate results (lower bounds, initial estimates, etc) is somewhat application-dependent.
NB: Callbacks that handle record updates should be insensitive to order: it is important not to modify attributes that might be used by later callbacks (e.g. the proposal.accepted flag).
- Attributes:
- params: Evaluation point for the function status: Status of the evaluation (pending, running, killed, completed) value: Return value (if completed) callbacks: Functions to call on status updates
-
add_callback
(callback)¶ Add a callback for update events.
-
cancel
()¶ Change status to ‘cancelled’ and execute callbacks.
-
complete
(value)¶ Change status to ‘completed’ and execute callbacks.
-
is_cancelled
¶ Check whether evaluation was cancelled
-
is_completed
¶ Check for successful completion of evaluation
-
is_done
¶ Check whether the status indicates the evaluation is finished.
-
is_killed
¶ Check whether evaluation is killed
-
is_pending
¶ Check if status is pending.
-
is_running
¶ Check if status is running.
-
kill
()¶ Change status to ‘killed’ and execute callbacks.
-
remove_callback
(callback)¶ Remove a callback subscriber.
-
running
()¶ Change status to ‘running’ and execute callbacks.
-
status
¶
-
update
(**kwargs)¶ Update fields and execute callbacks.
-
update_dict
(dict)¶ Update fields and execute callbacks.
- Args:
- dict: Dictionary of attributes to set on the record
-
class
GridCal.Engine.Replacements.strategy.
FixedSampleStrategy
(points)¶ Bases:
GridCal.Engine.Replacements.strategy.BaseStrategy
Sample at a fixed set of points.
The fixed sampling strategy is appropriate for any non-adaptive sampling scheme. Since the strategy is non-adaptive, we can employ as many available workers as we have points to process. We keep trying any evaluation that fails, and suggest termination only when all evaluations are complete. The points in the experimental design can be provided as any iterable object (e.g. a list or a generator function). One can use a generator for an infinite sequence if the fixed sampling strategy is used in combination with a strategy that provides a termination criterion.
-
propose_action
()¶ Propose an action based on outstanding points.
-
-
class
GridCal.Engine.Replacements.strategy.
InputStrategy
(controller, strategy)¶ Bases:
GridCal.Engine.Replacements.strategy.BaseStrategy
Insert requests from the outside world (e.g. from a GUI).
-
eval
(params, retry=True)¶ Request a new function evaluation
-
kill
(record, retry=False)¶ Request a function evaluation be killed
-
propose_action
()¶
-
terminate
(retry=True)¶ Request termination of the optimization
-
-
class
GridCal.Engine.Replacements.strategy.
MaxEvalStrategy
(controller, max_counter)¶ Bases:
object
Recommend termination of the iteration after some number of evals.
Recommends termination after observing some number of completed function evaluations. We allow more than the requisite number of evaluations to be started, since there’s always the possibility that one of our outstanding evaluations won’t finish.
- Attributes:
- counter: Number of completed evals max_counter: Max number of evals
-
on_new_feval
(record)¶ On every feval, add a callback.
-
on_update
(record)¶ On every completion, increment the counter.
-
propose_action
()¶ Propose termination once the eval counter is high enough.
-
class
GridCal.Engine.Replacements.strategy.
MultiStartStrategy
(controller, strategies)¶ Bases:
object
Merge several strategies by taking the first valid eval proposal.
This strategy is similar to the SimpleMergedStrategy, except that we only terminate once all the worker strategies have voted to terminate.
- Attributes:
- controller: Controller object used to determine whether we can eval strategies: Prioritized list of strategies
-
propose_action
()¶ Go through strategies in order and choose the first valid action. Terminate iff all workers vote to terminate.
-
class
GridCal.Engine.Replacements.strategy.
PromiseStrategy
(rvalue=<function PromiseStrategy.<lambda>>, block=True)¶ Bases:
GridCal.Engine.Replacements.strategy.BaseStrategy
Provides a promise-based asynchronous evaluation interface.
A promise (aka a future) is a common concurrent programming abstraction. A caller requests an asynchronous evaluation, and the call returns immediately with a promise object. The caller can check the promise object to see if it has a value ready, or wait for the value to become ready. The callee can set the value when it is ready.
- Attributes:
- proposalq: Queue of proposed actions caused by optimizer. valueq: Queue of function values from proposed actions. rvalue: Function to extract return value from a record block: Flag whether to block on proposal pop
-
class
Promise
(valueq)¶ Bases:
object
Evaluation promise.
- Properties:
- value: Promised value. Block on read if not ready.
-
ready
()¶ Check whether the value is ready (at consumer)
-
value
¶ Wait on the value (at consumer)
-
blocking_eval
(*args)¶ Request a function evaluation and block until done.
-
blocking_evals
(xs)¶ Request a list of function evaluations.
-
on_complete
(record)¶ Send the value to the consumer via the promise.
-
on_kill
(record)¶ Re-submit the proposal with the same promise.
-
on_reply_accept
(proposal)¶ Make sure we copy over the promise to the feval record.
-
on_reply_reject
(proposal)¶ Re-submit the proposal with the same promise.
-
on_terminate
()¶ Throw an exception at the consumer if still running on termination
-
on_terminate_reply_reject
(proposal)¶ Re-submit the termination proposal.
-
promise_eval
(*args)¶ Request a function evaluation and return a promise object.
-
propose_action
()¶ Provide proposals from the queue.
-
terminate
()¶ Request termination.
-
class
GridCal.Engine.Replacements.strategy.
Proposal
(action, *args)¶ Bases:
object
Represent a proposed action.
We currently recognize three types of proposals: evaluation requests (“eval”), requests to stop an evaluation (“kill”), and requests to terminate the optimization (“terminate”). When an evaluation proposal is accepted, the controller adds an evaluation record to the proposal before notifying subscribers.
NB: Callbacks that handle proposal accept/reject should be insensitive to order: it is important not to modify attributes that might be used by later callbacks (e.g. the proposal.accepted flag).
- Attributes:
- action: String describing the action args: Tuple of arguments to pass to the action accepted: Flag set on accept/reject decision callbacks: Functions to call on accept/reject of action record: Evaluation record for accepted evaluation proposals
-
accept
()¶ Mark the action as accepted and execute callbacks.
-
add_callback
(callback)¶ Add a callback subscriber to the action.
-
copy
()¶ Make a copy of the proposal.
-
reject
()¶ Mark action as rejected and execute callbacks.
-
remove_callback
(callback)¶ Remove a callback subscriber.
-
class
GridCal.Engine.Replacements.strategy.
RetryStrategy
¶ Bases:
GridCal.Engine.Replacements.strategy.BaseStrategy
Retry strategy class.
The RetryStrategy class manages a queue of proposals to be retried, either because they were rejected or because they correspond to a function evaluation that was killed.
If a proposal is submitted to a retry strategy and it has already been marked with the .retry attribute, we do not attempt to manage retries, as another object is assumed to have taken responsibility for retrying the proposal on failure. This prevents us from having redundant retries.
When a proposal is submitted to the retry class, we make a copy (the proposal draft) for safe-keeping. If at any point it is necessary to resubmit, we submit the draft copy to the retry class. The draft will have any modifications or callbacks that were added before the proposal reached this strategy, and none of those that were added afterward (including those added afterward by the RetryStrategy itself). The intent is that retried proposals inserted into the strategy hierarchy at the place where they were recorded should require no special treatment to avoid adding multiple copies of callbacks (for example).
- Attributes:
- proposals: Queue of outstanding proposals num_eval_pending: Number of pending evaluations num_eval_running: Number of running evaluations num_eval_outstanding: Number of outstanding evaluations
-
empty
()¶ Check if the queue is empty
-
get
()¶ Pop a proposal from the queue.
-
num_eval_outstanding
¶ Number of outstanding function evaluations.
-
on_complete
(record)¶ Clean up after completed eval+retry
-
on_kill
(record)¶ Resubmit proposal for killed or cancelled eval+retry
-
on_kill_reply_reject
(proposal)¶ Resubmit rejected kill+retry proposal
-
on_reply_accept
(proposal)¶ Process accepted eval+retry proposal
-
on_reply_reject
(proposal)¶ Resubmit rejected eval+retry proposal
-
on_terminate_reply_reject
(proposal)¶ Resubmit rejected termination+retry proposal
-
put
(proposal)¶ Put a non-retry proposal in the queue.
-
rput
(proposal)¶ Put a retry proposal in the queue.
-
exception
GridCal.Engine.Replacements.strategy.
RunTerminatedException
¶ Bases:
Exception
-
class
GridCal.Engine.Replacements.strategy.
SimpleMergedStrategy
(controller, strategies)¶ Bases:
object
Merge several strategies by taking the first valid proposal from them.
The simplest combination strategy is to keep a list of several possible strategies in some priority order, query them for proposals in priority order, and pass on the first plausible proposal to the controller.
- Attributes:
- controller: Controller object used to determine whether we can eval strategies: Prioritized list of strategies
-
propose_action
()¶ Go through strategies in order and choose the first valid action.
-
class
GridCal.Engine.Replacements.strategy.
ThreadStrategy
(controller, optimizer, rvalue=<function ThreadStrategy.<lambda>>)¶ Bases:
GridCal.Engine.Replacements.strategy.PromiseStrategy
Event-driven to serial adapter using threads.
The thread strategy runs a standard sequential optimization algorithm in a separate thread of control, with which the strategy communicates via promises. The optimizer thread intercepts function evaluation requests and completion and turns them into proposals for the strategy, which it places in a proposal queue. The optimizer then waits for the requested values (if any) to appear in the reply queue.
- Attributes:
- optimizer: Optimizer function (takes objective as an argument) proposalq: Queue of proposed actions caused by optimizer. valueq: Queue of function values from proposed actions. thread: Thread in which the optimizer runs. proposal: Proposal that the strategy is currently requesting. rvalue: Function mapping records to values
-
class
OptimizerThread
(strategy, optimizer)¶ Bases:
threading.Thread
-
run
()¶ Method representing the thread’s activity.
You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.
-
-
on_terminate
()¶ Throw an exception at the consumer if still running on termination
GridCal.Engine.Replacements.tcpserve module¶
-
class
GridCal.Engine.Replacements.tcpserve.
ProcessSocketWorker
(sockname, retries=0)¶ Bases:
GridCal.Engine.Replacements.tcpserve.SocketWorker
Socket worker that runs an evaluation in a subprocess
The ProcessSocketWorker is a base class for simulations that run a simulation in an external subprocess. This class provides functionality just to allow graceful termination of the external simulations.
- Attributes:
- process: Handle for external subprocess
-
kill
(record_id)¶ Kill a function evaluation
-
kill_process
()¶ Kill the child process
-
terminate
()¶ Terminate the worker
-
class
GridCal.Engine.Replacements.tcpserve.
SimpleSocketWorker
(objective, sockname, retries=0)¶ Bases:
GridCal.Engine.Replacements.tcpserve.SocketWorker
Simple socket worker that runs a local objective function
The SimpleSocketWorker is a socket worker that runs a local Python function and returns the result. It is probably mostly useful for testing – the ProcessSocketWorker is a better option for external simulations.
-
eval
(record_id, params)¶ Evaluate the function and send back a result.
If the function evaluation crashes, we send back a ‘cancel’ request for the record. If, on the other hand, there is a problem with calling send, we probably want to let the worker error out.
- Args:
- record_id: Feval record identifier used by server/controller params: Parameters sent to the function to be evaluated
-
-
class
GridCal.Engine.Replacements.tcpserve.
SocketWorker
(sockname, retries=0)¶ Bases:
object
Base class for workers that connect to SocketServer interface
The socket server interface is a server to which workers can connect. The socket worker is the client to that interface. It connects to a given TCP/IP port, then attempts to do work on the controller’s behalf.
- Attributes:
- running: True if the socket is active sock: Worker TCP socket
-
eval
(record_id, params)¶ Compute a function value
-
kill
(record_id)¶ Kill a function evaluation
-
marshall
(*args)¶ Marshall data to wire format
-
run
(loop=True)¶ Main loop
-
send
(*args)¶ Send a message to the controller
-
terminate
()¶ Terminate the worker
-
unmarshall
(data)¶ Convert data from wire format back to Python tuple
-
class
GridCal.Engine.Replacements.tcpserve.
SocketWorkerHandler
(request, client_address, server)¶ Bases:
socketserver.BaseRequestHandler
Manage a remote worker for a thread controller.
The SocketWorkerHandler is a request handler for incoming workers. It implements the socketserver request handler interface, and also the worker interface expected by the ThreadController.
-
eval
(record)¶ Send an evaluation request to remote worker
-
handle
()¶ Main event loop called from SocketServer
-
kill
(record)¶ Send a kill request to a remote worker
-
terminate
()¶ Send a termination request to a remote worker
-
-
class
GridCal.Engine.Replacements.tcpserve.
ThreadedTCPServer
(sockname=('localhost', 0), strategy=None, handlers={})¶ Bases:
socketserver.ThreadingMixIn
,socketserver.TCPServer
,object
SocketServer interface for workers to connect to controller.
The socket server interface lets workers connect to a given TCP/IP port and exchange updates with the controller.
The server sends messages of the form
(‘eval’, record_id, args, extra_args) (‘eval’, record_id, args) (‘kill’, record_id) (‘terminate’)The default messages received are
(‘running’, record_id) (‘kill’, record_id) (‘cancel’, record_id) (‘complete’, record_id, value)The set of handlers can also be extended with a dictionary of named callbacks to be invoked whenever a record update comes in. For example, to set a lower bound field, we might use the handler
- def set_lb(rec, value):
- rec.lb = value
handlers = {‘lb’ : set_lb }
This is useful for adding new types of updates without mucking around in the EvalRecord implementation.
- Attributes:
- controller: ThreadController that manages the optimization handlers: dictionary of specialized message handlers strategy: redirects to the controller strategy
-
marshall
(*args)¶ Convert an argument list to wire format.
-
run
(merit=<function ThreadedTCPServer.<lambda>>, filter=None)¶
-
sockname
¶
-
strategy
¶
-
unmarshall
(data)¶ Convert wire format back to Python arg list.