%% Generated by Sphinx.
\def\sphinxdocclass{report}
\documentclass[letterpaper,12pt,english]{sphinxmanual}
\ifdefined\pdfpxdimen
   \let\sphinxpxdimen\pdfpxdimen\else\newdimen\sphinxpxdimen
\fi \sphinxpxdimen=.75bp\relax
\ifdefined\pdfimageresolution
    \pdfimageresolution= \numexpr \dimexpr1in\relax/\sphinxpxdimen\relax
\fi
%% let collapsible pdf bookmarks panel have high depth per default
\PassOptionsToPackage{bookmarksdepth=5}{hyperref}

\PassOptionsToPackage{warn}{textcomp}
\usepackage[utf8]{inputenc}
\ifdefined\DeclareUnicodeCharacter
% support both utf8 and utf8x syntaxes
  \ifdefined\DeclareUnicodeCharacterAsOptional
    \def\sphinxDUC#1{\DeclareUnicodeCharacter{"#1}}
  \else
    \let\sphinxDUC\DeclareUnicodeCharacter
  \fi
  \sphinxDUC{00A0}{\nobreakspace}
  \sphinxDUC{2500}{\sphinxunichar{2500}}
  \sphinxDUC{2502}{\sphinxunichar{2502}}
  \sphinxDUC{2514}{\sphinxunichar{2514}}
  \sphinxDUC{251C}{\sphinxunichar{251C}}
  \sphinxDUC{2572}{\textbackslash}
\fi
\usepackage{cmap}
\usepackage[T1]{fontenc}
\usepackage{amsmath,amssymb,amstext}
\usepackage{babel}



\usepackage{tgtermes}
\usepackage{tgheros}
\renewcommand{\ttdefault}{txtt}



\usepackage[Bjarne]{fncychap}
\usepackage{sphinx}

\fvset{fontsize=auto}
\usepackage{geometry}


% Include hyperref last.
\usepackage{hyperref}
% Fix anchor placement for figures with captions.
\usepackage{hypcap}% it must be loaded after hyperref.
% Set up styles of URL: it should be placed after hyperref.
\urlstyle{same}

\addto\captionsenglish{\renewcommand{\contentsname}{Contents:}}

\usepackage{sphinxmessages}
\setcounter{tocdepth}{1}



\title{PARyOpt Documentation}
\date{Dec 20, 2021}
\release{1.0.2.1}
\author{Author}
\newcommand{\sphinxlogo}{\vbox{}}
\renewcommand{\releasename}{Release}
\makeindex
\begin{document}

\pagestyle{empty}
\sphinxmaketitle
\pagestyle{plain}
\sphinxtableofcontents
\pagestyle{normal}
\phantomsection\label{\detokenize{index::doc}}



\chapter{Introduction}
\label{\detokenize{introduction:introduction}}\label{\detokenize{introduction::doc}}
\sphinxAtStartPar
We consider a general minimization problem:
\begin{equation*}
\begin{split}\min_\mathbf{x} \, y(\mathbf{x})\end{split}
\end{equation*}
\sphinxAtStartPar
Bayesian optimization proceeds through construction of a surrogate cost
function \(\tilde{y}(\mathbf{x})\). This surrogate is represented as a basis
function expansion, around each evaluated point(\(\mathbf{x}_i, i \in [1,N]\)).
This ensures that the surrogate passes through (interpolates) the evaluated
points. In the case of evaluations with noisy data, the surrogate shall pass
within 1 standard deviation from the mean at the evaluated points.
Analytically, the surrogate \(\tilde{y}(\mathbf{x})\) after N function
evaluations is represented as
\begin{equation*}
\begin{split}\tilde{y}(\mathbf{x}) = \sum_{i=1}^N w_i k(\mathbf{x}, \mathbf{x}_i)\end{split}
\end{equation*}
\sphinxAtStartPar
where \(k(\mathbf{x}, \mathbf{x}_i)\) is a kernel function, i.e., it takes in two arguments,
\(\mathbf{x}, \, \mathbf{x}_i\) and returns a scalar value. This scalar is representative of how correlated is
the function \(y(\mathbf{x})\) at \(\mathbf{x}\) and \(\mathbf{x}_i\). The weights \(w_i\) are
calculated by solving the system of \(N\) linear equations in \(w_i\). In matrix notation, this is represented
using a covariance matrix(\(\mathbf{K}\)):
\begin{equation*}
\begin{split}\mathbf{K}\, \bar{w} & = y \\
\mathbf{K}_{i,j} & = k(\mathbf{x}_i, \mathbf{x}_j) , \, \, i,j\in[1,N] \\
y_i & = y(\mathbf{x}_i) , \, \,i\in[1,N] \\
\bar{w} & = \{w_i\}, \, \,i\in[1,N]\end{split}
\end{equation*}
\sphinxAtStartPar
Hence the weights are calculated through the inversion \(\bar{w} = \mathbf{K}^{-1}\,y\). Note that the covariance
matrix \(\mathbf{K}\) is a Gram matrix of a positive definite kernel function, making it symmetric and positive
semi\sphinxhyphen{}definite. Furthermore, since with every iteration only a finite number of rows are added to the covariance matrix,
efficient inversion is possible through incremental Cholesky decomposition. The mean and variance of the surrogate
are then calculated as:
\begin{equation*}
\begin{split}\mu(\mathbf{x}_{N+1}) & = \mathbf{k}^T \mathbf{K}^{-1} y_{1:N} \\
\sigma^2(\mathbf{x}_{N+1}) & = k(\mathbf{x}_{N+1}, \mathbf{x}_{N+1}) - \mathbf{k}^T\,\mathbf{K}^{-1}\,\mathbf{k}\end{split}
\end{equation*}
\sphinxAtStartPar
where
\begin{equation*}
\begin{split}\mathbf{k} = k(\mathbf{x}_{1:N}, \mathbf{x}_{N+1}) = [k(\mathbf{x}_1,\mathbf{x}_{N+1})\, k(\mathbf{x}_2,\mathbf{x}_{N+1})\, . . .            k(\mathbf{x}_N,\mathbf{x}_{N+1})]\end{split}
\end{equation*}
\sphinxAtStartPar
At each iteration, the surrogate is updated with new data from the cost function. The locations where the next
evaluation is done is determined through optimization of an \sphinxstyleemphasis{acquisition function}. An acquisition function is a means
to estimate the new information content at a location. It uses the mean and variance calculated in the above steps.

\sphinxAtStartPar
Some sample radial kernel functions include:
\begin{itemize}
\item {} 
\sphinxAtStartPar
squared exponential kernel function : Infinitely differentiable

\end{itemize}
\begin{equation*}
\begin{split}k(r) = \theta_0 exp\Bigg(- \frac{r^2}{\theta^2}\Bigg)\end{split}
\end{equation*}\begin{itemize}
\item {} 
\sphinxAtStartPar
Matern class of kernel function :

\end{itemize}
\begin{equation*}
\begin{split}k_{Matern}(r) = \frac{2^{1-\nu}}{\Gamma(\nu)}\Bigg(\frac{\sqrt{2\nu}r}{l}\Bigg)^\nu K_\nu\Bigg(\frac{\sqrt{2\nu}r}{l}\Bigg)\end{split}
\end{equation*}
\sphinxAtStartPar
where \(K_\nu\) is the modified Bessel function, \(\nu,l\) are positive constants
\begin{itemize}
\item {} 
\sphinxAtStartPar
Rational quadratic kernel function:

\end{itemize}
\begin{equation*}
\begin{split}k_{RQ}(r) = \Bigg(1 + \frac{r^2}{2\alpha l^2}\Bigg)^{-\alpha}\end{split}
\end{equation*}
\sphinxAtStartPar
where \(r = ||\mathbf{x}_1 - \mathbf{x}_2||\)

\sphinxAtStartPar
Some example acquisition functions are:
\begin{itemize}
\item {} 
\sphinxAtStartPar
Confidence bounds

\end{itemize}
\begin{equation*}
\begin{split}LCB = \mu - \kappa \sigma\end{split}
\end{equation*}\begin{itemize}
\item {} 
\sphinxAtStartPar
Probability of improvement

\end{itemize}
\begin{equation*}
\begin{split}PI = \mathbf{cdf}(\gamma)\end{split}
\end{equation*}\begin{itemize}
\item {} 
\sphinxAtStartPar
Expectation of improvement

\end{itemize}
\begin{equation*}
\begin{split}EI = sqrt(variance) * (\gamma * \mathbf{cdf}(\gamma) + \mathbf{pdf}(\gamma))\end{split}
\end{equation*}
\sphinxAtStartPar
where \(\gamma = \frac{\mu}{\sigma}\), \(\mathbf{cdf}\) is cumulative normal distribution function and
\(\mathbf{pdf}\) is normal probability distribution function


\chapter{Installation}
\label{\detokenize{installation:installation}}\label{\detokenize{installation::doc}}
\sphinxAtStartPar
PARyOpt requires Python 3.5 or above, NumPy, and SciPy for basic functionality.
Paramiko is required for cost functions evaluated on remote machines
(HPC clusters). Matplotlib is used for visualization for these examples,
but is not required.

\sphinxAtStartPar
These all will be installed when you do a

\begin{sphinxVerbatim}[commandchars=\\\{\}]
pip install paryopt
\end{sphinxVerbatim}
\begin{description}
\item[{(if you are on Ubuntu, you may need to do \sphinxcode{\sphinxupquote{sudo apt\sphinxhyphen{}get install python3\sphinxhyphen{}pip}}}] \leavevmode
\sphinxAtStartPar
and use \sphinxcode{\sphinxupquote{pip3}} here instead!)

\end{description}

\sphinxAtStartPar
Or, if you prefer an Anaconda environment:

\begin{sphinxVerbatim}[commandchars=\\\{\}]
conda create \PYGZhy{}n paryopt \PYG{n+nv}{python}\PYG{o}{=}\PYG{l+m}{3}.5 numpy scipy matplotlib paramiko
activate paryopt
\end{sphinxVerbatim}

\sphinxAtStartPar
Or, if you are using a manual download:

\begin{sphinxVerbatim}[commandchars=\\\{\}]
tar \PYGZhy{}xvf paryopt\PYGZhy{}1.0.2.1.tar.gz
\PYG{n+nb}{cd} paryopt/
python3.5 setup.py install
\end{sphinxVerbatim}

\sphinxAtStartPar
If you want a virtual environment (preferred), do:

\begin{sphinxVerbatim}[commandchars=\\\{\}]
tar \PYGZhy{}xvf paryopt\PYGZhy{}1.0.2.1.tar.gz
\PYG{n+nb}{cd} paryopt/
python3.5 \PYGZhy{}m venv venv
\PYG{n+nb}{source} venv/bin/activate
pip install \PYGZhy{}r requirements.txt
pip install setup.py
\end{sphinxVerbatim}


\chapter{PARyOpt package}
\label{\detokenize{PARyOpt:paryopt-package}}\label{\detokenize{PARyOpt::doc}}

\section{Subpackages}
\label{\detokenize{PARyOpt:subpackages}}

\subsection{PARyOpt.evaluators package}
\label{\detokenize{PARyOpt.evaluators:paryopt-evaluators-package}}\label{\detokenize{PARyOpt.evaluators::doc}}

\subsubsection{Submodules}
\label{\detokenize{PARyOpt.evaluators:submodules}}

\subsubsection{PARyOpt.evaluators.paryopt\_async module}
\label{\detokenize{PARyOpt.evaluators:module-PARyOpt.evaluators.paryopt_async}}\label{\detokenize{PARyOpt.evaluators:paryopt-evaluators-paryopt-async-module}}\index{module@\spxentry{module}!PARyOpt.evaluators.paryopt\_async@\spxentry{PARyOpt.evaluators.paryopt\_async}}\index{PARyOpt.evaluators.paryopt\_async@\spxentry{PARyOpt.evaluators.paryopt\_async}!module@\spxentry{module}}
\sphinxAtStartPar
Asynchronous evaluator super class
\index{AsyncFunctionEvaluator (class in PARyOpt.evaluators.paryopt\_async)@\spxentry{AsyncFunctionEvaluator}\spxextra{class in PARyOpt.evaluators.paryopt\_async}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.paryopt_async.AsyncFunctionEvaluator}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{class\DUrole{w}{  }}}\sphinxcode{\sphinxupquote{PARyOpt.evaluators.paryopt\_async.}}\sphinxbfcode{\sphinxupquote{AsyncFunctionEvaluator}}}{\emph{\DUrole{n}{required\_fraction}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{float}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{1.0}}, \emph{\DUrole{n}{max\_pending}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{int}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{0}}}{}
\sphinxAtStartPar
Bases: \sphinxcode{\sphinxupquote{object}}

\sphinxAtStartPar
Abstract base class for long\sphinxhyphen{}running cost functions (e.g. external simulations).
Must be subclassed. Subclasses should fill in start() and check\_for\_results().
Automatically saves state as jobs are submitted.
\index{check\_for\_result() (PARyOpt.evaluators.paryopt\_async.AsyncFunctionEvaluator method)@\spxentry{check\_for\_result()}\spxextra{PARyOpt.evaluators.paryopt\_async.AsyncFunctionEvaluator method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.paryopt_async.AsyncFunctionEvaluator.check_for_result}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{check\_for\_result}}}{\emph{\DUrole{n}{x}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{numpy.core.multiarray.array}}, \emph{\DUrole{n}{data}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{Any}}}{{ $\rightarrow$ Union\DUrole{p}{{[}}{\hyperref[\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.paryopt_async.ValueNotReady}]{\sphinxcrossref{PARyOpt.evaluators.paryopt\_async.ValueNotReady}}}\DUrole{p}{,}\DUrole{w}{  }{\hyperref[\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.paryopt_async.EvaluationFailed}]{\sphinxcrossref{PARyOpt.evaluators.paryopt\_async.EvaluationFailed}}}\DUrole{p}{,}\DUrole{w}{  }{\hyperref[\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.paryopt_async.EvaluateAgain}]{\sphinxcrossref{PARyOpt.evaluators.paryopt\_async.EvaluateAgain}}}\DUrole{p}{,}\DUrole{w}{  }float\DUrole{p}{{]}}}}
\sphinxAtStartPar
Returns the cost function evaluation at x, if the value is available. This method is only called after start.
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{x}} \textendash{} the point to evaluate the cost function at

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{data}} \textendash{} user data returned by start(x)

\end{itemize}

\item[{Returns}] \leavevmode
\sphinxAtStartPar
an instance of ValueNotReady if such, EvaluationFailed(reason), or the cost function value at x (float)

\end{description}\end{quote}

\end{fulllineitems}

\index{evaluate\_population() (PARyOpt.evaluators.paryopt\_async.AsyncFunctionEvaluator method)@\spxentry{evaluate\_population()}\spxextra{PARyOpt.evaluators.paryopt\_async.AsyncFunctionEvaluator method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.paryopt_async.AsyncFunctionEvaluator.evaluate_population}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{evaluate\_population}}}{\emph{\DUrole{n}{xs}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{List\DUrole{p}{{[}}numpy.core.multiarray.array\DUrole{p}{{]}}}}, \emph{\DUrole{n}{if\_ready\_xs}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{List\DUrole{p}{{[}}numpy.core.multiarray.array\DUrole{p}{{]}}}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{{[}{]}}}}{{ $\rightarrow$ Tuple\DUrole{p}{{[}}List\DUrole{p}{{[}}Tuple\DUrole{p}{{[}}numpy.core.multiarray.array\DUrole{p}{,}\DUrole{w}{  }float\DUrole{p}{{]}}\DUrole{p}{{]}}\DUrole{p}{,}\DUrole{w}{  }List\DUrole{p}{{[}}numpy.core.multiarray.array\DUrole{p}{{]}}\DUrole{p}{,}\DUrole{w}{  }List\DUrole{p}{{[}}numpy.core.multiarray.array\DUrole{p}{{]}}\DUrole{p}{{]}}}}
\sphinxAtStartPar
Evaluates a population of x values, encoded as a list of 1D numpy arrays.
Returns a tuple containing three lists:
\begin{itemize}
\item {} 
\sphinxAtStartPar
Completed values: {[} (x1, y1), (x2, y2), … {]}

\item {} 
\sphinxAtStartPar
Pending values \sphinxhyphen{} evaluation is in progress, but not complete: {[} x1, x2, … {]}

\item {} 
\sphinxAtStartPar
Failed  values \sphinxhyphen{} evaluation completed unsuccessfully: {[} x1, x2, … {]}

\end{itemize}

\sphinxAtStartPar
The union of completed, failed, and pending is equal to the union of xs and if\_ready\_xs.
The \_\_init\_\_ parameter required\_fraction tunes how many completed/pending values are returned.
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{xs}} \textendash{} list of new points to check

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{if\_ready\_xs}} \textendash{} List of points to include in the return tuple if they available by the time we
evaluate the minimum required percentage of xs. These points do not count towards
the minimum required completed points.

\end{itemize}

\item[{Returns}] \leavevmode
\sphinxAtStartPar
( {[}(x, y), …{]} completed, {[}x, …{]} pending, {[}x, …{]} failed )

\end{description}\end{quote}

\end{fulllineitems}

\index{start() (PARyOpt.evaluators.paryopt\_async.AsyncFunctionEvaluator method)@\spxentry{start()}\spxextra{PARyOpt.evaluators.paryopt\_async.AsyncFunctionEvaluator method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.paryopt_async.AsyncFunctionEvaluator.start}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{start}}}{\emph{\DUrole{n}{x}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{numpy.core.multiarray.array}}}{{ $\rightarrow$ Any}}
\sphinxAtStartPar
Start a cost function evaluation at the given x location.
This method may return anything \sphinxhyphen{} the data will be passed on to check\_for\_result().
The only restriction is that the return value should be pickle\sphinxhyphen{}able to enable restart support.
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{x}} \textendash{} point to begin evaluation at

\item[{Returns}] \leavevmode
\sphinxAtStartPar
user data that will be fed into check\_for\_result()

\end{description}\end{quote}

\end{fulllineitems}


\end{fulllineitems}

\index{EvaluateAgain (class in PARyOpt.evaluators.paryopt\_async)@\spxentry{EvaluateAgain}\spxextra{class in PARyOpt.evaluators.paryopt\_async}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.paryopt_async.EvaluateAgain}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{class\DUrole{w}{  }}}\sphinxcode{\sphinxupquote{PARyOpt.evaluators.paryopt\_async.}}\sphinxbfcode{\sphinxupquote{EvaluateAgain}}}{\emph{\DUrole{n}{reason}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{str}}}{}
\sphinxAtStartPar
Bases: \sphinxcode{\sphinxupquote{object}}

\sphinxAtStartPar
Indicates evaluation needs to be done again, due to some reason (for eg., during hardware failures)

\end{fulllineitems}

\index{EvaluationFailed (class in PARyOpt.evaluators.paryopt\_async)@\spxentry{EvaluationFailed}\spxextra{class in PARyOpt.evaluators.paryopt\_async}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.paryopt_async.EvaluationFailed}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{class\DUrole{w}{  }}}\sphinxcode{\sphinxupquote{PARyOpt.evaluators.paryopt\_async.}}\sphinxbfcode{\sphinxupquote{EvaluationFailed}}}{\emph{\DUrole{n}{reason}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{str}}}{}
\sphinxAtStartPar
Bases: \sphinxcode{\sphinxupquote{object}}

\sphinxAtStartPar
Indicates evaluation was not able to complete successfully, with an error value (i.e. an exception).

\end{fulllineitems}

\index{ValueNotReady (class in PARyOpt.evaluators.paryopt\_async)@\spxentry{ValueNotReady}\spxextra{class in PARyOpt.evaluators.paryopt\_async}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.paryopt_async.ValueNotReady}}\pysigline{\sphinxbfcode{\sphinxupquote{class\DUrole{w}{  }}}\sphinxcode{\sphinxupquote{PARyOpt.evaluators.paryopt\_async.}}\sphinxbfcode{\sphinxupquote{ValueNotReady}}}
\sphinxAtStartPar
Bases: \sphinxcode{\sphinxupquote{object}}

\sphinxAtStartPar
Indicates a function value is not ready yet.

\end{fulllineitems}



\subsubsection{PARyOpt.evaluators.async\_local module}
\label{\detokenize{PARyOpt.evaluators:module-PARyOpt.evaluators.async_local}}\label{\detokenize{PARyOpt.evaluators:paryopt-evaluators-async-local-module}}\index{module@\spxentry{module}!PARyOpt.evaluators.async\_local@\spxentry{PARyOpt.evaluators.async\_local}}\index{PARyOpt.evaluators.async\_local@\spxentry{PARyOpt.evaluators.async\_local}!module@\spxentry{module}}
\sphinxAtStartPar
Local asynchronous evaluator sub\sphinxhyphen{}class
\index{AsyncLocalEvaluator (class in PARyOpt.evaluators.async\_local)@\spxentry{AsyncLocalEvaluator}\spxextra{class in PARyOpt.evaluators.async\_local}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.async_local.AsyncLocalEvaluator}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{class\DUrole{w}{  }}}\sphinxcode{\sphinxupquote{PARyOpt.evaluators.async\_local.}}\sphinxbfcode{\sphinxupquote{AsyncLocalEvaluator}}}{\emph{\DUrole{n}{job\_generator}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{Callable\DUrole{p}{{[}}\DUrole{p}{{[}}str\DUrole{p}{,}\DUrole{w}{  }numpy.core.multiarray.array\DUrole{p}{{]}}\DUrole{p}{,}\DUrole{w}{  }None\DUrole{p}{{]}}}}, \emph{\DUrole{n}{run\_cmd\_generator}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{Callable\DUrole{p}{{[}}\DUrole{p}{{[}}str\DUrole{p}{,}\DUrole{w}{  }numpy.core.multiarray.array\DUrole{p}{{]}}\DUrole{p}{,}\DUrole{w}{  }Union\DUrole{p}{{[}}str\DUrole{p}{,}\DUrole{w}{  }List\DUrole{p}{{[}}Any\DUrole{p}{{]}}\DUrole{p}{{]}}\DUrole{p}{{]}}}}, \emph{\DUrole{n}{parse\_result}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{Callable\DUrole{p}{{[}}\DUrole{p}{{[}}str\DUrole{p}{,}\DUrole{w}{  }numpy.core.multiarray.array\DUrole{p}{{]}}\DUrole{p}{,}\DUrole{w}{  }float\DUrole{p}{{]}}}}, \emph{\DUrole{n}{jobs\_dir}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{str}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{\textquotesingle{}/home/balajip/PROJECTS/machineLearning/bayesianOptimization/paryopt/docs/opt\_jobs\textquotesingle{}}}, \emph{\DUrole{n}{required\_fraction}\DUrole{o}{=}\DUrole{default_value}{1.0}}, \emph{\DUrole{n}{max\_pending}\DUrole{o}{=}\DUrole{default_value}{8}}}{}
\sphinxAtStartPar
Bases: {\hyperref[\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.paryopt_async.AsyncFunctionEvaluator}]{\sphinxcrossref{\sphinxcode{\sphinxupquote{PARyOpt.evaluators.paryopt\_async.AsyncFunctionEvaluator}}}}}

\sphinxAtStartPar
Class for cost functions that evaluated by launching a long\sphinxhyphen{}running process on the local machine.
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{job\_generator}} \textendash{} callable that sets up the run directory for a given x (by e.g. writing config files).     It will be passed two arguments: the job directory and the point to evaluate at (x).

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{run\_cmd\_generator}} \textendash{} callable that returns the command to run the job.     It will be passed two arguments: the job directory and the point to evaluate at (x).     If run\_cmd\_generator returns a string, the string will be run by the default shell (typically /bin/sh)     via Popen with shell=True. If run\_cmd\_generator returns a list, it will be passed to Popen.     In both cases, the CWD is set to the job directory.

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{parse\_result}} \textendash{} callable that returns the cost function evaluated at x.     It will be passed two arguments: the job directory and the point to evaluate a t (x).    This will be called after the command returned by run\_cmd\_generator has terminated (gracefully or otherwise).     If the process did not terminate successfully or the result is otherwise unavailable, parse\_result     should raise any exception. This will signal the optimization routine to not try this point again.

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{jobs\_dir}} \textendash{} optional base directory to run jobs in \sphinxhyphen{} default is \$PWD/opt\_jobs.

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{required\_fraction}} \textendash{} fraction of points which must complete before continuing to the next iteration see     AsyncEvaluator for more info and implementation

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{max\_pending}} \textendash{} maximum simultaneous processes, defaults to multiprocessing.cpu\_count() see AsyncEvaluator for     implementation

\end{itemize}

\end{description}\end{quote}
\index{check\_for\_result() (PARyOpt.evaluators.async\_local.AsyncLocalEvaluator method)@\spxentry{check\_for\_result()}\spxextra{PARyOpt.evaluators.async\_local.AsyncLocalEvaluator method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.async_local.AsyncLocalEvaluator.check_for_result}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{check\_for\_result}}}{\emph{\DUrole{n}{x: numpy.core.multiarray.array}}, \emph{\DUrole{n}{data: (\textless{}class \textquotesingle{}str\textquotesingle{}\textgreater{}}}, \emph{\DUrole{n}{\textless{}class \textquotesingle{}int\textquotesingle{}\textgreater{})}}}{{ $\rightarrow$ Union\DUrole{p}{{[}}{\hyperref[\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.paryopt_async.ValueNotReady}]{\sphinxcrossref{PARyOpt.evaluators.paryopt\_async.ValueNotReady}}}\DUrole{p}{,}\DUrole{w}{  }{\hyperref[\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.paryopt_async.EvaluationFailed}]{\sphinxcrossref{PARyOpt.evaluators.paryopt\_async.EvaluationFailed}}}\DUrole{p}{,}\DUrole{w}{  }{\hyperref[\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.paryopt_async.EvaluateAgain}]{\sphinxcrossref{PARyOpt.evaluators.paryopt\_async.EvaluateAgain}}}\DUrole{p}{,}\DUrole{w}{  }float\DUrole{p}{{]}}}}
\sphinxAtStartPar
Checks the status of pid, in data, and calls parse\_result if the job is done.
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{x}} \textendash{} location of function evaluation

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{data}} \textendash{} list of directory and pid

\end{itemize}

\item[{Returns}] \leavevmode
\sphinxAtStartPar
either ValueNotReady float or EvaluationFailed()

\end{description}\end{quote}

\end{fulllineitems}

\index{start() (PARyOpt.evaluators.async\_local.AsyncLocalEvaluator method)@\spxentry{start()}\spxextra{PARyOpt.evaluators.async\_local.AsyncLocalEvaluator method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.async_local.AsyncLocalEvaluator.start}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{start}}}{\emph{\DUrole{n}{x: numpy.core.multiarray.array) \sphinxhyphen{}\textgreater{} (\textless{}class \textquotesingle{}str\textquotesingle{}\textgreater{}}}, \emph{\DUrole{n}{\textless{}class \textquotesingle{}int\textquotesingle{}\textgreater{}}}}{}
\sphinxAtStartPar
Start a cost function evaluation at the given x location.
This method may return anything \sphinxhyphen{} the data will be passed on to check\_for\_result().
The only restriction is that the return value should be pickle\sphinxhyphen{}able to enable restart support.
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{x}} \textendash{} point to begin evaluation at

\item[{Returns}] \leavevmode
\sphinxAtStartPar
user data that will be fed into check\_for\_result()

\end{description}\end{quote}

\end{fulllineitems}


\end{fulllineitems}



\subsubsection{PARyOpt.evaluators.async\_parse\_result\_local module}
\label{\detokenize{PARyOpt.evaluators:module-PARyOpt.evaluators.async_parse_result_local}}\label{\detokenize{PARyOpt.evaluators:paryopt-evaluators-async-parse-result-local-module}}\index{module@\spxentry{module}!PARyOpt.evaluators.async\_parse\_result\_local@\spxentry{PARyOpt.evaluators.async\_parse\_result\_local}}\index{PARyOpt.evaluators.async\_parse\_result\_local@\spxentry{PARyOpt.evaluators.async\_parse\_result\_local}!module@\spxentry{module}}
\sphinxAtStartPar
AsyncLocalParseResultEvaluator
An evaluator that only parses files periodically for function evaluations and
does not actually execute a script. It periodically checks for the file “job\_folder/if\_parse.txt” 
whether the function evaluation is complete or not. If the file contains anything other than ‘False’, 
it assumes the external function evaluation is completed and hence checks for “job\_folder/y.txt” 
for the cost function value.
\index{AsyncLocalParseResultEvaluator (class in PARyOpt.evaluators.async\_parse\_result\_local)@\spxentry{AsyncLocalParseResultEvaluator}\spxextra{class in PARyOpt.evaluators.async\_parse\_result\_local}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.async_parse_result_local.AsyncLocalParseResultEvaluator}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{class\DUrole{w}{  }}}\sphinxcode{\sphinxupquote{PARyOpt.evaluators.async\_parse\_result\_local.}}\sphinxbfcode{\sphinxupquote{AsyncLocalParseResultEvaluator}}}{\emph{\DUrole{n}{parse\_result}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{Optional\DUrole{p}{{[}}Callable\DUrole{p}{{[}}\DUrole{p}{{[}}str\DUrole{p}{,}\DUrole{w}{  }numpy.core.multiarray.array\DUrole{p}{{]}}\DUrole{p}{,}\DUrole{w}{  }float\DUrole{p}{{]}}\DUrole{p}{{]}}}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{None}}, \emph{\DUrole{n}{job\_generator}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{Optional\DUrole{p}{{[}}Callable\DUrole{p}{{[}}\DUrole{p}{{[}}str\DUrole{p}{,}\DUrole{w}{  }numpy.core.multiarray.array\DUrole{p}{{]}}\DUrole{p}{,}\DUrole{w}{  }None\DUrole{p}{{]}}\DUrole{p}{{]}}}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{None}}, \emph{\DUrole{n}{jobs\_dir}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{str}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{\textquotesingle{}/home/balajip/PROJECTS/machineLearning/bayesianOptimization/paryopt/docs/opt\_jobs\textquotesingle{}}}, \emph{\DUrole{n}{wait\_time}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{datetime.timedelta}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{datetime.timedelta(0, 60)}}, \emph{\DUrole{n}{total\_folders}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{int}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{16}}, \emph{\DUrole{n}{max\_pending}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{int}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{0}}, \emph{\DUrole{n}{required\_fraction}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{float}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{1.0}}}{}
\sphinxAtStartPar
Bases: {\hyperref[\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.paryopt_async.AsyncFunctionEvaluator}]{\sphinxcrossref{\sphinxcode{\sphinxupquote{PARyOpt.evaluators.paryopt\_async.AsyncFunctionEvaluator}}}}}

\sphinxAtStartPar
Fills files in a set of folders and periodically checks if the external evaluator has finished     evaluation. Supports asynchronous evaluations. Only on a local machine.
\begin{quote}\begin{description}
\item[{Variables}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{folder\_num}} \textendash{} folder into which the location values are written into

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{job\_generator}} \textendash{} callable that sets up the run directory for a given x (by e.g. writing config files).     It will be passed two arguments: the job directory and the point to evaluate at (x).

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{jobs\_dir}} \textendash{} base directory where the jobs are written into

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{parse\_result}} \textendash{} function to parse result from directory , if not specified, it will search in     jobs\_dir/folder\_\textless{}folder\_num\textgreater{}/if\_parse.txt and y.txt

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{total\_folders}} \textendash{} total number of folders to write into

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{wait\_time}} \textendash{} min time to wait before parsing the results folder

\end{itemize}

\end{description}\end{quote}
\index{check\_for\_result() (PARyOpt.evaluators.async\_parse\_result\_local.AsyncLocalParseResultEvaluator method)@\spxentry{check\_for\_result()}\spxextra{PARyOpt.evaluators.async\_parse\_result\_local.AsyncLocalParseResultEvaluator method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.async_parse_result_local.AsyncLocalParseResultEvaluator.check_for_result}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{check\_for\_result}}}{\emph{\DUrole{n}{x: numpy.core.multiarray.array}}, \emph{\DUrole{n}{data: (\textless{}class \textquotesingle{}str\textquotesingle{}\textgreater{}}}, \emph{\DUrole{n}{\textless{}class \textquotesingle{}datetime.datetime\textquotesingle{}\textgreater{})}}}{{ $\rightarrow$ Union\DUrole{p}{{[}}{\hyperref[\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.paryopt_async.ValueNotReady}]{\sphinxcrossref{PARyOpt.evaluators.paryopt\_async.ValueNotReady}}}\DUrole{p}{,}\DUrole{w}{  }{\hyperref[\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.paryopt_async.EvaluationFailed}]{\sphinxcrossref{PARyOpt.evaluators.paryopt\_async.EvaluationFailed}}}\DUrole{p}{,}\DUrole{w}{  }{\hyperref[\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.paryopt_async.EvaluateAgain}]{\sphinxcrossref{PARyOpt.evaluators.paryopt\_async.EvaluateAgain}}}\DUrole{p}{,}\DUrole{w}{  }float\DUrole{p}{{]}}}}
\sphinxAtStartPar
Function to check if a location has been evaluated or not (ValueNotReady). If it completes, categorize the         result as one of a float , EvaluationFailed or EvaluateAgain
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{x}} \textendash{} location to evaluate

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{data}} \textendash{} directory

\end{itemize}

\item[{Returns}] \leavevmode
\sphinxAtStartPar
Union{[}ValueNotReady, EvaluationFailed, EvaluateAgain, float{]}

\end{description}\end{quote}

\end{fulllineitems}

\index{start() (PARyOpt.evaluators.async\_parse\_result\_local.AsyncLocalParseResultEvaluator method)@\spxentry{start()}\spxextra{PARyOpt.evaluators.async\_parse\_result\_local.AsyncLocalParseResultEvaluator method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.async_parse_result_local.AsyncLocalParseResultEvaluator.start}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{start}}}{\emph{\DUrole{n}{x: numpy.core.multiarray.array) \sphinxhyphen{}\textgreater{} (\textless{}class \textquotesingle{}str\textquotesingle{}\textgreater{}}}, \emph{\DUrole{n}{\textless{}class \textquotesingle{}datetime.datetime\textquotesingle{}\textgreater{}}}}{}
\sphinxAtStartPar
function to start a cost function evaluation (write to file in this case) given the location         of evaluation
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{x}} \textendash{} location of evaluation

\item[{Returns}] \leavevmode
\sphinxAtStartPar
folder name in which it was submitted

\end{description}\end{quote}

\end{fulllineitems}


\end{fulllineitems}



\subsubsection{PARyOpt.evaluators.async\_sbatch module}
\label{\detokenize{PARyOpt.evaluators:module-PARyOpt.evaluators.async_sbatch}}\label{\detokenize{PARyOpt.evaluators:paryopt-evaluators-async-sbatch-module}}\index{module@\spxentry{module}!PARyOpt.evaluators.async\_sbatch@\spxentry{PARyOpt.evaluators.async\_sbatch}}\index{PARyOpt.evaluators.async\_sbatch@\spxentry{PARyOpt.evaluators.async\_sbatch}!module@\spxentry{module}}
\sphinxAtStartPar
SLURM scheduler asynchronous evaluator sub\sphinxhyphen{}class
\index{AsyncSbatchEvaluator (class in PARyOpt.evaluators.async\_sbatch)@\spxentry{AsyncSbatchEvaluator}\spxextra{class in PARyOpt.evaluators.async\_sbatch}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.async_sbatch.AsyncSbatchEvaluator}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{class\DUrole{w}{  }}}\sphinxcode{\sphinxupquote{PARyOpt.evaluators.async\_sbatch.}}\sphinxbfcode{\sphinxupquote{AsyncSbatchEvaluator}}}{\emph{\DUrole{n}{host}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{{\hyperref[\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.connection.Host}]{\sphinxcrossref{PARyOpt.evaluators.connection.Host}}}}}, \emph{\DUrole{n}{job\_generator}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{Callable\DUrole{p}{{[}}\DUrole{p}{{[}}str\DUrole{p}{,}\DUrole{w}{  }numpy.core.multiarray.array\DUrole{p}{{]}}\DUrole{p}{,}\DUrole{w}{  }None\DUrole{p}{{]}}}}, \emph{\DUrole{n}{job\_script}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{Union\DUrole{p}{{[}}str\DUrole{p}{,}\DUrole{w}{  }Callable\DUrole{p}{{[}}\DUrole{p}{{[}}str\DUrole{p}{,}\DUrole{w}{  }numpy.core.multiarray.array\DUrole{p}{{]}}\DUrole{p}{,}\DUrole{w}{  }str\DUrole{p}{{]}}\DUrole{p}{{]}}}}, \emph{\DUrole{n}{lcl\_parse\_result}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{Optional\DUrole{p}{{[}}Callable\DUrole{p}{{[}}\DUrole{p}{{[}}str\DUrole{p}{,}\DUrole{w}{  }str\DUrole{p}{,}\DUrole{w}{  }{\hyperref[\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.connection.Connection}]{\sphinxcrossref{PARyOpt.evaluators.connection.Connection}}}\DUrole{p}{,}\DUrole{w}{  }numpy.core.multiarray.array\DUrole{p}{{]}}\DUrole{p}{,}\DUrole{w}{  }float\DUrole{p}{{]}}\DUrole{p}{{]}}}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{None}}, \emph{\DUrole{n}{remote\_parse\_result}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{Optional\DUrole{p}{{[}}Callable\DUrole{p}{{[}}\DUrole{p}{{[}}str\DUrole{p}{,}\DUrole{w}{  }numpy.core.multiarray.array\DUrole{p}{{]}}\DUrole{p}{,}\DUrole{w}{  }float\DUrole{p}{{]}}\DUrole{p}{{]}}}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{None}}, \emph{\DUrole{n}{lcl\_jobs\_dir}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{str}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{\textquotesingle{}/home/balajip/PROJECTS/machineLearning/bayesianOptimization/paryopt/docs/opt\_jobs\textquotesingle{}}}, \emph{\DUrole{n}{squeue\_update\_rate}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{datetime.timedelta}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{datetime.timedelta(0, 30)}}, \emph{\DUrole{n}{remote\_jobs\_dir}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{str}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{\textquotesingle{}paryopt\_jobs\textquotesingle{}}}, \emph{\DUrole{n}{required\_fraction}\DUrole{o}{=}\DUrole{default_value}{1.0}}, \emph{\DUrole{n}{max\_pending}\DUrole{o}{=}\DUrole{default_value}{25}}}{}
\sphinxAtStartPar
Bases: {\hyperref[\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.paryopt_async.AsyncFunctionEvaluator}]{\sphinxcrossref{\sphinxcode{\sphinxupquote{PARyOpt.evaluators.paryopt\_async.AsyncFunctionEvaluator}}}}}

\sphinxAtStartPar
Class for cost functions that evaluated by launching a job on a remote machine running the SLURM job scheduler.
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{host}} \textendash{} Host object containing the credentials for the server to connect to

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{job\_generator}} \textendash{} callable that sets up the run directory for a given x (by e.g. writing config files).     It will be passed two arguments: the job directory and the point to evaluate at (x).

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{job\_script}} \textendash{} either a string (for a fixed job script), or a callable that returns the job script string.     In the latter case, job\_script will be passed two arguments: the job directory and the point to evaluate at (x).

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{remote\_parse\_result}} \textendash{} callable that returns the cost function evaluated at x.     It will be passed two arguments: the job directory and the point to evaluate at (x).     This will be called after the command returned by run\_cmd\_generator has terminated (gracefully or otherwise).     If the process did not terminate successfully or the result is otherwise unavailable, parse\_result     should raise any exception. This will signal the optimization routine to not try this point again.     This function will be executed \sphinxstyleemphasis{on the remote host}. This requires the remote host to have a matching version     of Python installed and the dill module.

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{lcl\_parse\_result}} \textendash{} callable that returns the cost function evaluated at X.     It is passed three arguments: the local job dir, remote job dir, the Connection object to the remote, and X.     It is executed on the local machine. This does not require the remote to have Python installed.

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{lcl\_jobs\_dir}} \textendash{} optional base directory to generate jobs in \sphinxhyphen{} default is \$PWD/opt\_jobs.

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{remote\_jobs\_dir}} \textendash{} optional base directory to upload jobs to \sphinxhyphen{} default is \$HOME/paryopt\_jobs.

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{squeue\_update\_rate}} \textendash{} minimum time between squeue calls. Lower for better job latency, higher to be more     polite

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{required\_fraction}} \textendash{} fraction of points which must complete before continuing to the next iteration see     AsyncEvaluator for more info and implementation

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{max\_pending}} \textendash{} maximum simultaneous queued jobs, defaults to 25, see AsyncEvaluator for implementation

\end{itemize}

\end{description}\end{quote}
\index{check\_for\_result() (PARyOpt.evaluators.async\_sbatch.AsyncSbatchEvaluator method)@\spxentry{check\_for\_result()}\spxextra{PARyOpt.evaluators.async\_sbatch.AsyncSbatchEvaluator method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.async_sbatch.AsyncSbatchEvaluator.check_for_result}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{check\_for\_result}}}{\emph{\DUrole{n}{x: numpy.core.multiarray.array}}, \emph{\DUrole{n}{data: (\textless{}class \textquotesingle{}str\textquotesingle{}\textgreater{}}}, \emph{\DUrole{n}{\textless{}class \textquotesingle{}str\textquotesingle{}\textgreater{}}}, \emph{\DUrole{n}{\textless{}class \textquotesingle{}int\textquotesingle{}\textgreater{}}}, \emph{\DUrole{n}{\textless{}class \textquotesingle{}datetime.datetime\textquotesingle{}\textgreater{})}}}{{ $\rightarrow$ Union\DUrole{p}{{[}}{\hyperref[\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.paryopt_async.ValueNotReady}]{\sphinxcrossref{PARyOpt.evaluators.paryopt\_async.ValueNotReady}}}\DUrole{p}{,}\DUrole{w}{  }{\hyperref[\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.paryopt_async.EvaluateAgain}]{\sphinxcrossref{PARyOpt.evaluators.paryopt\_async.EvaluateAgain}}}\DUrole{p}{,}\DUrole{w}{  }{\hyperref[\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.paryopt_async.EvaluationFailed}]{\sphinxcrossref{PARyOpt.evaluators.paryopt\_async.EvaluationFailed}}}\DUrole{p}{,}\DUrole{w}{  }float\DUrole{p}{{]}}}}
\sphinxAtStartPar
checks for result if the jobid is complete and ping time is after update rate
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{x}} \textendash{} location of evaluation

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{data}} \textendash{} data related to the location. Typically this is directory information, job id and submit time

\end{itemize}

\item[{Returns}] \leavevmode
\sphinxAtStartPar
one of ValueNotReady, EvaluateAgain, EvaluationFailed or float

\end{description}\end{quote}

\end{fulllineitems}

\index{squeue() (PARyOpt.evaluators.async\_sbatch.AsyncSbatchEvaluator method)@\spxentry{squeue()}\spxextra{PARyOpt.evaluators.async\_sbatch.AsyncSbatchEvaluator method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.async_sbatch.AsyncSbatchEvaluator.squeue}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{squeue}}}{}{}
\end{fulllineitems}

\index{start() (PARyOpt.evaluators.async\_sbatch.AsyncSbatchEvaluator method)@\spxentry{start()}\spxextra{PARyOpt.evaluators.async\_sbatch.AsyncSbatchEvaluator method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.async_sbatch.AsyncSbatchEvaluator.start}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{start}}}{\emph{\DUrole{n}{x: numpy.core.multiarray.array) \sphinxhyphen{}\textgreater{} (\textless{}class \textquotesingle{}str\textquotesingle{}\textgreater{}}}, \emph{\DUrole{n}{\textless{}class \textquotesingle{}str\textquotesingle{}\textgreater{}}}, \emph{\DUrole{n}{\textless{}class \textquotesingle{}int\textquotesingle{}\textgreater{}}}, \emph{\DUrole{n}{\textless{}class \textquotesingle{}datetime.datetime\textquotesingle{}\textgreater{}}}}{}
\sphinxAtStartPar
Generate job directory on local machine, fill in data related to the job like directory, job id and submit time

\end{fulllineitems}


\end{fulllineitems}

\index{VALUE\_FROM\_FILE() (in module PARyOpt.evaluators.async\_sbatch)@\spxentry{VALUE\_FROM\_FILE()}\spxextra{in module PARyOpt.evaluators.async\_sbatch}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.async_sbatch.VALUE_FROM_FILE}}\pysiglinewithargsret{\sphinxcode{\sphinxupquote{PARyOpt.evaluators.async\_sbatch.}}\sphinxbfcode{\sphinxupquote{VALUE\_FROM\_FILE}}}{\emph{\DUrole{n}{filename}}}{}
\end{fulllineitems}



\subsubsection{PARyOpt.evaluators.connection module}
\label{\detokenize{PARyOpt.evaluators:module-PARyOpt.evaluators.connection}}\label{\detokenize{PARyOpt.evaluators:paryopt-evaluators-connection-module}}\index{module@\spxentry{module}!PARyOpt.evaluators.connection@\spxentry{PARyOpt.evaluators.connection}}\index{PARyOpt.evaluators.connection@\spxentry{PARyOpt.evaluators.connection}!module@\spxentry{module}}\index{Connection (class in PARyOpt.evaluators.connection)@\spxentry{Connection}\spxextra{class in PARyOpt.evaluators.connection}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.connection.Connection}}\pysigline{\sphinxbfcode{\sphinxupquote{class\DUrole{w}{  }}}\sphinxcode{\sphinxupquote{PARyOpt.evaluators.connection.}}\sphinxbfcode{\sphinxupquote{Connection}}}
\sphinxAtStartPar
Bases: \sphinxcode{\sphinxupquote{object}}
\index{call\_on\_remote() (PARyOpt.evaluators.connection.Connection method)@\spxentry{call\_on\_remote()}\spxextra{PARyOpt.evaluators.connection.Connection method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.connection.Connection.call_on_remote}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{call\_on\_remote}}}{\emph{\DUrole{n}{remote\_func}}, \emph{\DUrole{o}{*}\DUrole{n}{args}}, \emph{\DUrole{n}{remote\_cwd}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{Optional\DUrole{p}{{[}}str\DUrole{p}{{]}}}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{None}}}{}
\sphinxAtStartPar
Call a function created on this system on a remote system with args.
This is done by pickling it with dill, SFTPing it to a file on the remote, executing a Python script on the
remote that un\sphinxhyphen{}dills the file, calls the function, dills the result and prints it to stdout.
Finally, stdout is un\sphinxhyphen{}dilled on the local machine to give the return value.
This requires the remote to have a matching Python version.
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{remote\_func}} \textendash{} function to call

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{args}} \textendash{} any arguments to call the function with

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{remote\_cwd}} \textendash{} directory on the remote to call the script from (must have write access to this directory)

\end{itemize}

\item[{Returns}] \leavevmode
\sphinxAtStartPar
value returned by f

\end{description}\end{quote}

\end{fulllineitems}

\index{connect() (PARyOpt.evaluators.connection.Connection method)@\spxentry{connect()}\spxextra{PARyOpt.evaluators.connection.Connection method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.connection.Connection.connect}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{connect}}}{\emph{\DUrole{n}{host}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{{\hyperref[\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.connection.Host}]{\sphinxcrossref{PARyOpt.evaluators.connection.Host}}}}}}{}
\end{fulllineitems}

\index{exec\_command() (PARyOpt.evaluators.connection.Connection method)@\spxentry{exec\_command()}\spxextra{PARyOpt.evaluators.connection.Connection method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.connection.Connection.exec_command}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{exec\_command}}}{\emph{\DUrole{n}{cmd: str}}, \emph{\DUrole{n}{cwd=None}}, \emph{\DUrole{n}{check\_exitcode=True}}, \emph{\DUrole{n}{encoding=\textquotesingle{}utf\sphinxhyphen{}8\textquotesingle{}) \sphinxhyphen{}\textgreater{} (\textless{}class \textquotesingle{}str\textquotesingle{}\textgreater{}}}, \emph{\DUrole{n}{\textless{}class \textquotesingle{}str\textquotesingle{}\textgreater{}}}, \emph{\DUrole{n}{\textless{}class \textquotesingle{}int\textquotesingle{}\textgreater{}}}}{}
\sphinxAtStartPar
Executes cmd in a new shell session on the remote host.
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{cmd}} \textendash{} command to execute

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{cwd}} \textendash{} directory to execute the command in \sphinxhyphen{} performed by prepending ‘cd {[}cwd{]} \&\& ‘ to cmd

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{check\_exitcode}} \textendash{} if true, instead of returning the exit code of cmd as part of the return tuple, verify
that the return code is zero. If it is not, an exception is raised with the contents of stderr.

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{encoding}} \textendash{} encoding to decode stdout/stderr with. Defaults to utf\sphinxhyphen{}8.

\end{itemize}

\item[{Returns}] \leavevmode
\sphinxAtStartPar
if check\_exitcode is True, (stdout: str, stderr: str). If it is False, (stdout, stderr, rc: int).
stdout and stderr are decoded according to encoding.

\end{description}\end{quote}

\end{fulllineitems}

\index{get\_dir() (PARyOpt.evaluators.connection.Connection method)@\spxentry{get\_dir()}\spxextra{PARyOpt.evaluators.connection.Connection method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.connection.Connection.get_dir}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{get\_dir}}}{\emph{\DUrole{n}{remote\_dir}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{str}}, \emph{\DUrole{n}{local\_dir}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{str}}}{{ $\rightarrow$ None}}
\sphinxAtStartPar
Download directory from remote directory to local directory

\end{fulllineitems}

\index{get\_file() (PARyOpt.evaluators.connection.Connection method)@\spxentry{get\_file()}\spxextra{PARyOpt.evaluators.connection.Connection method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.connection.Connection.get_file}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{get\_file}}}{\emph{\DUrole{n}{remote\_path}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{str}}, \emph{\DUrole{n}{local\_path}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{str}}}{{ $\rightarrow$ None}}
\sphinxAtStartPar
Downloads a file from the remote, same as paramiko.SFTPClient.get

\end{fulllineitems}

\index{mkdirs() (PARyOpt.evaluators.connection.Connection method)@\spxentry{mkdirs()}\spxextra{PARyOpt.evaluators.connection.Connection method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.connection.Connection.mkdirs}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{mkdirs}}}{\emph{\DUrole{n}{remote\_dir}}}{{ $\rightarrow$ None}}
\sphinxAtStartPar
Creates remote\_dir recursively as a directory on the remote, creating any necessaries parent directories
along the way. Similar to \sphinxtitleref{mkdir \sphinxhyphen{}p}, except it doesn’t error if the directory already exists.

\end{fulllineitems}

\index{put\_dir() (PARyOpt.evaluators.connection.Connection method)@\spxentry{put\_dir()}\spxextra{PARyOpt.evaluators.connection.Connection method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.connection.Connection.put_dir}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{put\_dir}}}{\emph{\DUrole{n}{local\_path}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{str}}, \emph{\DUrole{n}{remote\_path}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{str}}}{{ $\rightarrow$ None}}
\sphinxAtStartPar
Compresses local\_path into a .tar.gz archive, uploads it to the remote, extracts it into remote\_path,
and finally deletes the temporary tar archive. Assumes the remote has the ‘tar’ utility available.

\end{fulllineitems}

\index{put\_file() (PARyOpt.evaluators.connection.Connection method)@\spxentry{put\_file()}\spxextra{PARyOpt.evaluators.connection.Connection method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.connection.Connection.put_file}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{put\_file}}}{\emph{\DUrole{n}{local\_path}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{str}}, \emph{\DUrole{n}{remote\_path}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{str}}}{{ $\rightarrow$ None}}
\sphinxAtStartPar
Uploads a local file to the remote host, same as paramiko.SFTPClient.put

\end{fulllineitems}

\index{remote\_python() (PARyOpt.evaluators.connection.Connection method)@\spxentry{remote\_python()}\spxextra{PARyOpt.evaluators.connection.Connection method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.connection.Connection.remote_python}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{remote\_python}}}{}{{ $\rightarrow$ str}}
\sphinxAtStartPar
Returns a string that, when invoked as a command on the remote, will execute a Python that:
\begin{itemize}
\item {} 
\sphinxAtStartPar
Matches the version that this script was invoked with (i.e. matching sys.version\_info)

\item {} 
\sphinxAtStartPar
Has the ‘dill’ module installed

\end{itemize}

\sphinxAtStartPar
The remote Python is discovered by trial and error using common Python names.
The search is performed once and then cached.
If no such Python is available, this will return None.

\end{fulllineitems}

\index{sftp() (PARyOpt.evaluators.connection.Connection method)@\spxentry{sftp()}\spxextra{PARyOpt.evaluators.connection.Connection method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.connection.Connection.sftp}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{sftp}}}{}{{ $\rightarrow$ paramiko.sftp\_client.SFTPClient}}
\end{fulllineitems}


\end{fulllineitems}

\index{Host (class in PARyOpt.evaluators.connection)@\spxentry{Host}\spxextra{class in PARyOpt.evaluators.connection}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.connection.Host}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{class\DUrole{w}{  }}}\sphinxcode{\sphinxupquote{PARyOpt.evaluators.connection.}}\sphinxbfcode{\sphinxupquote{Host}}}{\emph{\DUrole{n}{username}}, \emph{\DUrole{n}{hostname}}, \emph{\DUrole{n}{port}\DUrole{o}{=}\DUrole{default_value}{22}}}{}
\sphinxAtStartPar
Bases: \sphinxcode{\sphinxupquote{object}}
\index{get\_interactive() (PARyOpt.evaluators.connection.Host method)@\spxentry{get\_interactive()}\spxextra{PARyOpt.evaluators.connection.Host method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.connection.Host.get_interactive}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{get\_interactive}}}{\emph{\DUrole{n}{title}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{str}}, \emph{\DUrole{n}{instructions}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{str}}, \emph{\DUrole{n}{prompts}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{List\DUrole{p}{{[}}str\DUrole{p}{{]}}}}}{{ $\rightarrow$ List\DUrole{p}{{[}}str\DUrole{p}{{]}}}}
\sphinxAtStartPar
Handles the ssh ‘interactive’ authentication mode (user answers a series of prompts).
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{title}} \textendash{} title of the window

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{instructions}} \textendash{} instructions, to be shown before any prompts

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{prompts}} \textendash{} the list of prompts

\end{itemize}

\item[{Returns}] \leavevmode
\sphinxAtStartPar
the list of responses (in the same order as the prompts)

\end{description}\end{quote}

\end{fulllineitems}

\index{get\_keys() (PARyOpt.evaluators.connection.Host method)@\spxentry{get\_keys()}\spxextra{PARyOpt.evaluators.connection.Host method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.connection.Host.get_keys}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{get\_keys}}}{}{}~\begin{quote}\begin{description}
\item[{Returns}] \leavevmode
\sphinxAtStartPar
a list of public/private keys to try authenticating with

\end{description}\end{quote}

\end{fulllineitems}

\index{get\_password() (PARyOpt.evaluators.connection.Host method)@\spxentry{get\_password()}\spxextra{PARyOpt.evaluators.connection.Host method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.connection.Host.get_password}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{get\_password}}}{}{}~\begin{quote}\begin{description}
\item[{Returns}] \leavevmode
\sphinxAtStartPar
the password for the host

\end{description}\end{quote}

\end{fulllineitems}


\end{fulllineitems}



\subsubsection{Module contents}
\label{\detokenize{PARyOpt.evaluators:module-PARyOpt.evaluators}}\label{\detokenize{PARyOpt.evaluators:module-contents}}\index{module@\spxentry{module}!PARyOpt.evaluators@\spxentry{PARyOpt.evaluators}}\index{PARyOpt.evaluators@\spxentry{PARyOpt.evaluators}!module@\spxentry{module}}\index{FunctionEvaluator (class in PARyOpt.evaluators)@\spxentry{FunctionEvaluator}\spxextra{class in PARyOpt.evaluators}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.FunctionEvaluator}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{class\DUrole{w}{  }}}\sphinxcode{\sphinxupquote{PARyOpt.evaluators.}}\sphinxbfcode{\sphinxupquote{FunctionEvaluator}}}{\emph{\DUrole{n}{func}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{Callable\DUrole{p}{{[}}numpy.core.multiarray.array\DUrole{p}{,}\DUrole{w}{  }float\DUrole{p}{{]}}}}}{}
\sphinxAtStartPar
Bases: \sphinxcode{\sphinxupquote{object}}

\sphinxAtStartPar
The simplest function evaluator \sphinxhyphen{} evaluates a Python function for each point.
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{func}} \textendash{} cost function to evaluate at each x

\end{description}\end{quote}
\index{evaluate\_population() (PARyOpt.evaluators.FunctionEvaluator method)@\spxentry{evaluate\_population()}\spxextra{PARyOpt.evaluators.FunctionEvaluator method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.evaluators:PARyOpt.evaluators.FunctionEvaluator.evaluate_population}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{evaluate\_population}}}{\emph{\DUrole{n}{xs}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{List\DUrole{p}{{[}}numpy.core.multiarray.array\DUrole{p}{{]}}}}, \emph{\DUrole{n}{old\_xs}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{List\DUrole{p}{{[}}numpy.core.multiarray.array\DUrole{p}{{]}}}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{{[}{]}}}}{{ $\rightarrow$ Tuple\DUrole{p}{{[}}List\DUrole{p}{{[}}Tuple\DUrole{p}{{[}}numpy.core.multiarray.array\DUrole{p}{,}\DUrole{w}{  }float\DUrole{p}{{]}}\DUrole{p}{{]}}\DUrole{p}{,}\DUrole{w}{  }List\DUrole{p}{{[}}numpy.core.multiarray.array\DUrole{p}{{]}}\DUrole{p}{,}\DUrole{w}{  }List\DUrole{p}{{[}}numpy.core.multiarray.array\DUrole{p}{{]}}\DUrole{p}{{]}}}}
\end{fulllineitems}


\end{fulllineitems}



\subsection{PARyOpt.kernel package}
\label{\detokenize{PARyOpt.kernel:paryopt-kernel-package}}\label{\detokenize{PARyOpt.kernel::doc}}

\subsubsection{Submodules}
\label{\detokenize{PARyOpt.kernel:submodules}}

\subsubsection{PARyOpt.kernel.kernel\_function module}
\label{\detokenize{PARyOpt.kernel:module-PARyOpt.kernel.kernel_function}}\label{\detokenize{PARyOpt.kernel:paryopt-kernel-kernel-function-module}}\index{module@\spxentry{module}!PARyOpt.kernel.kernel\_function@\spxentry{PARyOpt.kernel.kernel\_function}}\index{PARyOpt.kernel.kernel\_function@\spxentry{PARyOpt.kernel.kernel\_function}!module@\spxentry{module}}\index{KernelFunction (class in PARyOpt.kernel.kernel\_function)@\spxentry{KernelFunction}\spxextra{class in PARyOpt.kernel.kernel\_function}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.kernel:PARyOpt.kernel.kernel_function.KernelFunction}}\pysigline{\sphinxbfcode{\sphinxupquote{class\DUrole{w}{  }}}\sphinxcode{\sphinxupquote{PARyOpt.kernel.kernel\_function.}}\sphinxbfcode{\sphinxupquote{KernelFunction}}}
\sphinxAtStartPar
Bases: \sphinxcode{\sphinxupquote{object}}

\sphinxAtStartPar
Attributes:
\begin{quote}\begin{description}
\item[{Variables}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{theta}} \textendash{} Kernel parameters. Can be None, a scalar, or a vector. Should be initialized to the appropriate     type/len during initialization.

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{theta0}} \textendash{} Scaling factor

\end{itemize}

\end{description}\end{quote}
\index{derivative() (PARyOpt.kernel.kernel\_function.KernelFunction method)@\spxentry{derivative()}\spxextra{PARyOpt.kernel.kernel\_function.KernelFunction method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.kernel:PARyOpt.kernel.kernel_function.KernelFunction.derivative}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{derivative}}}{\emph{\DUrole{n}{x1}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{numpy.core.multiarray.array}}, \emph{\DUrole{n}{x2}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{numpy.core.multiarray.array}}}{{ $\rightarrow$ numpy.core.multiarray.array}}
\sphinxAtStartPar
Evaluate the derivative of the kernel
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{x1}} \textendash{} location 1

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{x2}} \textendash{} location 2

\end{itemize}

\item[{Returns}] \leavevmode
\sphinxAtStartPar


\end{description}\end{quote}

\end{fulllineitems}

\index{eval() (PARyOpt.kernel.kernel\_function.KernelFunction method)@\spxentry{eval()}\spxextra{PARyOpt.kernel.kernel\_function.KernelFunction method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.kernel:PARyOpt.kernel.kernel_function.KernelFunction.eval}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{eval}}}{\emph{\DUrole{n}{x1}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{numpy.core.multiarray.array}}, \emph{\DUrole{n}{x2}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{numpy.core.multiarray.array}}}{{ $\rightarrow$ float}}
\sphinxAtStartPar
Evaluate the kernel function
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{x1}} \textendash{} location 1

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{x2}} \textendash{} location 2

\end{itemize}

\item[{Returns}] \leavevmode
\sphinxAtStartPar


\end{description}\end{quote}

\end{fulllineitems}

\index{theta (PARyOpt.kernel.kernel\_function.KernelFunction attribute)@\spxentry{theta}\spxextra{PARyOpt.kernel.kernel\_function.KernelFunction attribute}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.kernel:PARyOpt.kernel.kernel_function.KernelFunction.theta}}\pysigline{\sphinxbfcode{\sphinxupquote{theta}}\sphinxbfcode{\sphinxupquote{\DUrole{w}{  }\DUrole{p}{=}\DUrole{w}{  }None}}}
\end{fulllineitems}

\index{theta0 (PARyOpt.kernel.kernel\_function.KernelFunction attribute)@\spxentry{theta0}\spxextra{PARyOpt.kernel.kernel\_function.KernelFunction attribute}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.kernel:PARyOpt.kernel.kernel_function.KernelFunction.theta0}}\pysigline{\sphinxbfcode{\sphinxupquote{theta0}}\sphinxbfcode{\sphinxupquote{\DUrole{w}{  }\DUrole{p}{=}\DUrole{w}{  }1.0}}}
\end{fulllineitems}


\end{fulllineitems}



\subsubsection{PARyOpt.kernel.matern module}
\label{\detokenize{PARyOpt.kernel:module-PARyOpt.kernel.matern}}\label{\detokenize{PARyOpt.kernel:paryopt-kernel-matern-module}}\index{module@\spxentry{module}!PARyOpt.kernel.matern@\spxentry{PARyOpt.kernel.matern}}\index{PARyOpt.kernel.matern@\spxentry{PARyOpt.kernel.matern}!module@\spxentry{module}}\index{Matern32 (class in PARyOpt.kernel.matern)@\spxentry{Matern32}\spxextra{class in PARyOpt.kernel.matern}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.kernel:PARyOpt.kernel.matern.Matern32}}\pysigline{\sphinxbfcode{\sphinxupquote{class\DUrole{w}{  }}}\sphinxcode{\sphinxupquote{PARyOpt.kernel.matern.}}\sphinxbfcode{\sphinxupquote{Matern32}}}
\sphinxAtStartPar
Bases: {\hyperref[\detokenize{PARyOpt.kernel:PARyOpt.kernel.kernel_function.KernelFunction}]{\sphinxcrossref{\sphinxcode{\sphinxupquote{PARyOpt.kernel.kernel\_function.KernelFunction}}}}}

\sphinxAtStartPar
3/2 matern function rbf
\index{derivative() (PARyOpt.kernel.matern.Matern32 method)@\spxentry{derivative()}\spxextra{PARyOpt.kernel.matern.Matern32 method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.kernel:PARyOpt.kernel.matern.Matern32.derivative}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{derivative}}}{\emph{\DUrole{n}{x1}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{numpy.core.multiarray.array}}, \emph{\DUrole{n}{x2}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{numpy.core.multiarray.array}}}{{ $\rightarrow$ numpy.core.multiarray.array}}
\sphinxAtStartPar
Evaluate the derivative of the kernel
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{x1}} \textendash{} location 1

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{x2}} \textendash{} location 2

\end{itemize}

\item[{Returns}] \leavevmode
\sphinxAtStartPar


\end{description}\end{quote}

\end{fulllineitems}

\index{eval() (PARyOpt.kernel.matern.Matern32 method)@\spxentry{eval()}\spxextra{PARyOpt.kernel.matern.Matern32 method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.kernel:PARyOpt.kernel.matern.Matern32.eval}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{eval}}}{\emph{\DUrole{n}{x1}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{numpy.core.multiarray.array}}, \emph{\DUrole{n}{x2}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{numpy.core.multiarray.array}}}{{ $\rightarrow$ float}}
\sphinxAtStartPar
Evaluate the kernel function
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{x1}} \textendash{} location 1

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{x2}} \textendash{} location 2

\end{itemize}

\item[{Returns}] \leavevmode
\sphinxAtStartPar


\end{description}\end{quote}

\end{fulllineitems}

\index{theta (PARyOpt.kernel.matern.Matern32 attribute)@\spxentry{theta}\spxextra{PARyOpt.kernel.matern.Matern32 attribute}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.kernel:PARyOpt.kernel.matern.Matern32.theta}}\pysigline{\sphinxbfcode{\sphinxupquote{theta}}\sphinxbfcode{\sphinxupquote{\DUrole{w}{  }\DUrole{p}{=}\DUrole{w}{  }1.0}}}
\end{fulllineitems}


\end{fulllineitems}

\index{Matern52 (class in PARyOpt.kernel.matern)@\spxentry{Matern52}\spxextra{class in PARyOpt.kernel.matern}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.kernel:PARyOpt.kernel.matern.Matern52}}\pysigline{\sphinxbfcode{\sphinxupquote{class\DUrole{w}{  }}}\sphinxcode{\sphinxupquote{PARyOpt.kernel.matern.}}\sphinxbfcode{\sphinxupquote{Matern52}}}
\sphinxAtStartPar
Bases: {\hyperref[\detokenize{PARyOpt.kernel:PARyOpt.kernel.kernel_function.KernelFunction}]{\sphinxcrossref{\sphinxcode{\sphinxupquote{PARyOpt.kernel.kernel\_function.KernelFunction}}}}}

\sphinxAtStartPar
5/2 matern function rbf
\index{derivative() (PARyOpt.kernel.matern.Matern52 method)@\spxentry{derivative()}\spxextra{PARyOpt.kernel.matern.Matern52 method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.kernel:PARyOpt.kernel.matern.Matern52.derivative}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{derivative}}}{\emph{\DUrole{n}{x1}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{numpy.core.multiarray.array}}, \emph{\DUrole{n}{x2}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{numpy.core.multiarray.array}}}{{ $\rightarrow$ numpy.core.multiarray.array}}
\sphinxAtStartPar
Evaluate the derivative of the kernel
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{x1}} \textendash{} location 1

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{x2}} \textendash{} location 2

\end{itemize}

\item[{Returns}] \leavevmode
\sphinxAtStartPar


\end{description}\end{quote}

\end{fulllineitems}

\index{eval() (PARyOpt.kernel.matern.Matern52 method)@\spxentry{eval()}\spxextra{PARyOpt.kernel.matern.Matern52 method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.kernel:PARyOpt.kernel.matern.Matern52.eval}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{eval}}}{\emph{\DUrole{n}{x1}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{numpy.core.multiarray.array}}, \emph{\DUrole{n}{x2}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{numpy.core.multiarray.array}}}{{ $\rightarrow$ float}}
\sphinxAtStartPar
Evaluate the kernel function
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{x1}} \textendash{} location 1

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{x2}} \textendash{} location 2

\end{itemize}

\item[{Returns}] \leavevmode
\sphinxAtStartPar


\end{description}\end{quote}

\end{fulllineitems}

\index{theta (PARyOpt.kernel.matern.Matern52 attribute)@\spxentry{theta}\spxextra{PARyOpt.kernel.matern.Matern52 attribute}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.kernel:PARyOpt.kernel.matern.Matern52.theta}}\pysigline{\sphinxbfcode{\sphinxupquote{theta}}\sphinxbfcode{\sphinxupquote{\DUrole{w}{  }\DUrole{p}{=}\DUrole{w}{  }1.0}}}
\end{fulllineitems}


\end{fulllineitems}



\subsubsection{PARyOpt.kernel.squared\_exponential module}
\label{\detokenize{PARyOpt.kernel:module-PARyOpt.kernel.squared_exponential}}\label{\detokenize{PARyOpt.kernel:paryopt-kernel-squared-exponential-module}}\index{module@\spxentry{module}!PARyOpt.kernel.squared\_exponential@\spxentry{PARyOpt.kernel.squared\_exponential}}\index{PARyOpt.kernel.squared\_exponential@\spxentry{PARyOpt.kernel.squared\_exponential}!module@\spxentry{module}}\index{SquaredExponential (class in PARyOpt.kernel.squared\_exponential)@\spxentry{SquaredExponential}\spxextra{class in PARyOpt.kernel.squared\_exponential}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.kernel:PARyOpt.kernel.squared_exponential.SquaredExponential}}\pysigline{\sphinxbfcode{\sphinxupquote{class\DUrole{w}{  }}}\sphinxcode{\sphinxupquote{PARyOpt.kernel.squared\_exponential.}}\sphinxbfcode{\sphinxupquote{SquaredExponential}}}
\sphinxAtStartPar
Bases: {\hyperref[\detokenize{PARyOpt.kernel:PARyOpt.kernel.kernel_function.KernelFunction}]{\sphinxcrossref{\sphinxcode{\sphinxupquote{PARyOpt.kernel.kernel\_function.KernelFunction}}}}}

\sphinxAtStartPar
Squared exponential rbf
\index{derivative() (PARyOpt.kernel.squared\_exponential.SquaredExponential method)@\spxentry{derivative()}\spxextra{PARyOpt.kernel.squared\_exponential.SquaredExponential method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.kernel:PARyOpt.kernel.squared_exponential.SquaredExponential.derivative}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{derivative}}}{\emph{\DUrole{n}{x1}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{numpy.core.multiarray.array}}, \emph{\DUrole{n}{x2}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{numpy.core.multiarray.array}}}{{ $\rightarrow$ numpy.core.multiarray.array}}
\sphinxAtStartPar
Evaluate the derivative of the kernel
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{x1}} \textendash{} location 1

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{x2}} \textendash{} location 2

\end{itemize}

\item[{Returns}] \leavevmode
\sphinxAtStartPar


\end{description}\end{quote}

\end{fulllineitems}

\index{eval() (PARyOpt.kernel.squared\_exponential.SquaredExponential method)@\spxentry{eval()}\spxextra{PARyOpt.kernel.squared\_exponential.SquaredExponential method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.kernel:PARyOpt.kernel.squared_exponential.SquaredExponential.eval}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{eval}}}{\emph{\DUrole{n}{x1}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{numpy.core.multiarray.array}}, \emph{\DUrole{n}{x2}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{numpy.core.multiarray.array}}}{{ $\rightarrow$ float}}
\sphinxAtStartPar
Evaluate the kernel function
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{x1}} \textendash{} location 1

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{x2}} \textendash{} location 2

\end{itemize}

\item[{Returns}] \leavevmode
\sphinxAtStartPar


\end{description}\end{quote}

\end{fulllineitems}

\index{theta (PARyOpt.kernel.squared\_exponential.SquaredExponential attribute)@\spxentry{theta}\spxextra{PARyOpt.kernel.squared\_exponential.SquaredExponential attribute}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt.kernel:PARyOpt.kernel.squared_exponential.SquaredExponential.theta}}\pysigline{\sphinxbfcode{\sphinxupquote{theta}}\sphinxbfcode{\sphinxupquote{\DUrole{w}{  }\DUrole{p}{=}\DUrole{w}{  }1.0}}}
\end{fulllineitems}


\end{fulllineitems}



\subsubsection{Module contents}
\label{\detokenize{PARyOpt.kernel:module-PARyOpt.kernel}}\label{\detokenize{PARyOpt.kernel:module-contents}}\index{module@\spxentry{module}!PARyOpt.kernel@\spxentry{PARyOpt.kernel}}\index{PARyOpt.kernel@\spxentry{PARyOpt.kernel}!module@\spxentry{module}}

\section{Submodules}
\label{\detokenize{PARyOpt:submodules}}

\section{PARyOpt.acquisition\_functions module}
\label{\detokenize{PARyOpt:module-PARyOpt.acquisition_functions}}\label{\detokenize{PARyOpt:paryopt-acquisition-functions-module}}\index{module@\spxentry{module}!PARyOpt.acquisition\_functions@\spxentry{PARyOpt.acquisition\_functions}}\index{PARyOpt.acquisition\_functions@\spxentry{PARyOpt.acquisition\_functions}!module@\spxentry{module}}
\sphinxAtStartPar
Contains a library of acquisition functions that can be used in bayesian optimization
\index{expected\_improvement() (in module PARyOpt.acquisition\_functions)@\spxentry{expected\_improvement()}\spxextra{in module PARyOpt.acquisition\_functions}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt:PARyOpt.acquisition_functions.expected_improvement}}\pysiglinewithargsret{\sphinxcode{\sphinxupquote{PARyOpt.acquisition\_functions.}}\sphinxbfcode{\sphinxupquote{expected\_improvement}}}{\emph{\DUrole{n}{mean}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{float}}, \emph{\DUrole{n}{variance}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{float}}, \emph{\DUrole{n}{curr\_best}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{float}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{0.0}}, \emph{\DUrole{n}{\_}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{float}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{1.0}}}{{ $\rightarrow$ float}}
\sphinxAtStartPar
Expected improvement of objective function     ‘A Tutorial on Bayesian Optimization of Expensive Cost Functions,     with Application to Active User Modeling and Hierarchical Reinforcement Learning’
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{mean}} \textendash{} mean of surrogate

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{variance}} \textendash{} variance of surrogate

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{curr\_best}} \textendash{} current best evaluated point

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{kappa}} \textendash{} exploration \sphinxhyphen{} exploitation tradeoff parameter

\end{itemize}

\item[{Returns}] \leavevmode
\sphinxAtStartPar
expectation of improvement

\end{description}\end{quote}

\end{fulllineitems}

\index{lower\_confidence\_bound() (in module PARyOpt.acquisition\_functions)@\spxentry{lower\_confidence\_bound()}\spxextra{in module PARyOpt.acquisition\_functions}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt:PARyOpt.acquisition_functions.lower_confidence_bound}}\pysiglinewithargsret{\sphinxcode{\sphinxupquote{PARyOpt.acquisition\_functions.}}\sphinxbfcode{\sphinxupquote{lower\_confidence\_bound}}}{\emph{\DUrole{n}{mean}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{float}}, \emph{\DUrole{n}{variance}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{float}}, \emph{\DUrole{n}{curr\_best}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{float}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{0.0}}, \emph{\DUrole{n}{kappa}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{float}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{1.0}}}{{ $\rightarrow$ float}}
\sphinxAtStartPar
lower confidence bound of improvement : used for minimization problems
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{mean}} \textendash{} mean of surrogate

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{variance}} \textendash{} variance of surrogate

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{curr\_best}} \textendash{} current best evaluated point

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{kappa}} \textendash{} exploration \sphinxhyphen{} exploitation tradeoff parameter

\end{itemize}

\item[{Returns}] \leavevmode
\sphinxAtStartPar
lower confidence bound

\end{description}\end{quote}

\end{fulllineitems}

\index{probability\_improvement() (in module PARyOpt.acquisition\_functions)@\spxentry{probability\_improvement()}\spxextra{in module PARyOpt.acquisition\_functions}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt:PARyOpt.acquisition_functions.probability_improvement}}\pysiglinewithargsret{\sphinxcode{\sphinxupquote{PARyOpt.acquisition\_functions.}}\sphinxbfcode{\sphinxupquote{probability\_improvement}}}{\emph{\DUrole{n}{mean}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{float}}, \emph{\DUrole{n}{variance}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{float}}, \emph{\DUrole{n}{curr\_best}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{float}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{0.0}}, \emph{\DUrole{n}{\_}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{float}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{1.0}}}{{ $\rightarrow$ float}}
\sphinxAtStartPar
Probability of improvement of objective function     ‘A Tutorial on Bayesian Optimization of Expensive Cost Functions,     with Application to Active User Modeling and Hierarchical Reinforcement Learning’
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{mean}} \textendash{} mean of surrogate

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{variance}} \textendash{} variance of surrogate

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{curr\_best}} \textendash{} current best evaluated point

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{kappa}} \textendash{} exploration \sphinxhyphen{} exploitation tradeoff parameter

\end{itemize}

\item[{Returns}] \leavevmode
\sphinxAtStartPar
probability of improvement

\end{description}\end{quote}

\end{fulllineitems}

\index{upper\_confidence\_bound() (in module PARyOpt.acquisition\_functions)@\spxentry{upper\_confidence\_bound()}\spxextra{in module PARyOpt.acquisition\_functions}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt:PARyOpt.acquisition_functions.upper_confidence_bound}}\pysiglinewithargsret{\sphinxcode{\sphinxupquote{PARyOpt.acquisition\_functions.}}\sphinxbfcode{\sphinxupquote{upper\_confidence\_bound}}}{\emph{\DUrole{n}{mean}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{float}}, \emph{\DUrole{n}{variance}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{float}}, \emph{\DUrole{n}{curr\_best}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{float}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{0.0}}, \emph{\DUrole{n}{kappa}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{float}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{1.0}}}{{ $\rightarrow$ float}}
\sphinxAtStartPar
upper confidence bound of improvement: used in the case of maximization problems
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{mean}} \textendash{} mean of surrogate

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{variance}} \textendash{} variance of surrogate

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{curr\_best}} \textendash{} current best evaluated point

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{kappa}} \textendash{} exploration \sphinxhyphen{} exploitation tradeoff parameter

\end{itemize}

\item[{Returns}] \leavevmode
\sphinxAtStartPar
upper confidence bound

\end{description}\end{quote}

\end{fulllineitems}



\section{PARyOpt.paryopt module}
\label{\detokenize{PARyOpt:module-PARyOpt.paryopt}}\label{\detokenize{PARyOpt:paryopt-paryopt-module}}\index{module@\spxentry{module}!PARyOpt.paryopt@\spxentry{PARyOpt.paryopt}}\index{PARyOpt.paryopt@\spxentry{PARyOpt.paryopt}!module@\spxentry{module}}
\sphinxAtStartPar
Main class for bayesian optimization
\index{BayesOpt (class in PARyOpt.paryopt)@\spxentry{BayesOpt}\spxextra{class in PARyOpt.paryopt}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt:PARyOpt.paryopt.BayesOpt}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{class\DUrole{w}{  }}}\sphinxcode{\sphinxupquote{PARyOpt.paryopt.}}\sphinxbfcode{\sphinxupquote{BayesOpt}}}{\emph{\DUrole{n}{cost\_function}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{Union\DUrole{p}{{[}}Callable\DUrole{p}{{[}}List\DUrole{p}{{[}}numpy.core.multiarray.array\DUrole{p}{{]}}\DUrole{p}{,}\DUrole{w}{  }Tuple\DUrole{p}{{[}}List\DUrole{p}{{[}}Tuple\DUrole{p}{{[}}numpy.core.multiarray.array\DUrole{p}{,}\DUrole{w}{  }float\DUrole{p}{{]}}\DUrole{p}{{]}}\DUrole{p}{,}\DUrole{w}{  }List\DUrole{p}{{[}}numpy.core.multiarray.array\DUrole{p}{{]}}\DUrole{p}{,}\DUrole{w}{  }List\DUrole{p}{{[}}numpy.core.multiarray.array\DUrole{p}{{]}}\DUrole{p}{{]}}\DUrole{p}{{]}}\DUrole{p}{,}\DUrole{w}{  }Callable\DUrole{p}{{[}}\DUrole{p}{{[}}List\DUrole{p}{{[}}numpy.core.multiarray.array\DUrole{p}{{]}}\DUrole{p}{,}\DUrole{w}{  }List\DUrole{p}{{[}}numpy.core.multiarray.array\DUrole{p}{{]}}\DUrole{p}{{]}}\DUrole{p}{,}\DUrole{w}{  }Tuple\DUrole{p}{{[}}List\DUrole{p}{{[}}Tuple\DUrole{p}{{[}}numpy.core.multiarray.array\DUrole{p}{,}\DUrole{w}{  }float\DUrole{p}{{]}}\DUrole{p}{{]}}\DUrole{p}{,}\DUrole{w}{  }List\DUrole{p}{{[}}numpy.core.multiarray.array\DUrole{p}{{]}}\DUrole{p}{,}\DUrole{w}{  }List\DUrole{p}{{[}}numpy.core.multiarray.array\DUrole{p}{{]}}\DUrole{p}{{]}}\DUrole{p}{{]}}\DUrole{p}{{]}}}}, \emph{\DUrole{n}{l\_bound}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{numpy.core.multiarray.array}}, \emph{\DUrole{n}{u\_bound}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{numpy.core.multiarray.array}}, \emph{\DUrole{n}{n\_dim}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{int}}, \emph{\DUrole{n}{n\_opt}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{int}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{1}}, \emph{\DUrole{n}{n\_init}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{int}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{0}}, \emph{\DUrole{n}{init\_strategy}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{Optional\DUrole{p}{{[}}Callable\DUrole{p}{{[}}\DUrole{p}{{[}}int\DUrole{p}{,}\DUrole{w}{  }int\DUrole{p}{,}\DUrole{w}{  }numpy.core.multiarray.array\DUrole{p}{,}\DUrole{w}{  }numpy.core.multiarray.array\DUrole{p}{{]}}\DUrole{p}{,}\DUrole{w}{  }List\DUrole{p}{{[}}numpy.core.multiarray.array\DUrole{p}{{]}}\DUrole{p}{{]}}\DUrole{p}{{]}}}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{None}}, \emph{\DUrole{n}{do\_init}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{bool}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{True}}, \emph{\DUrole{n}{kern\_function}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{Optional\DUrole{p}{{[}}Union\DUrole{p}{{[}}str\DUrole{p}{,}\DUrole{w}{  }{\hyperref[\detokenize{PARyOpt.kernel:PARyOpt.kernel.kernel_function.KernelFunction}]{\sphinxcrossref{PARyOpt.kernel.kernel\_function.KernelFunction}}}\DUrole{p}{{]}}\DUrole{p}{{]}}}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{None}}, \emph{\DUrole{n}{acq\_func}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{Optional\DUrole{p}{{[}}Union\DUrole{p}{{[}}str\DUrole{p}{,}\DUrole{w}{  }Callable\DUrole{p}{{[}}\DUrole{p}{{[}}float\DUrole{p}{,}\DUrole{w}{  }float\DUrole{p}{,}\DUrole{w}{  }float\DUrole{p}{,}\DUrole{w}{  }float\DUrole{p}{{]}}\DUrole{p}{,}\DUrole{w}{  }float\DUrole{p}{{]}}\DUrole{p}{{]}}\DUrole{p}{{]}}}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{None}}, \emph{\DUrole{n}{acq\_func\_optimizer}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{Optional\DUrole{p}{{[}}Callable\DUrole{p}{{]}}}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{None}}, \emph{\DUrole{n}{kappa\_strategy}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{Optional\DUrole{p}{{[}}Union\DUrole{p}{{[}}List\DUrole{p}{{[}}Callable\DUrole{p}{{[}}int\DUrole{p}{,}\DUrole{w}{  }float\DUrole{p}{{]}}\DUrole{p}{{]}}\DUrole{p}{,}\DUrole{w}{  }Callable\DUrole{p}{{[}}int\DUrole{p}{,}\DUrole{w}{  }float\DUrole{p}{{]}}\DUrole{p}{{]}}\DUrole{p}{{]}}}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{None}}, \emph{\DUrole{n}{constraints}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{List\DUrole{p}{{[}}Callable\DUrole{p}{{[}}numpy.core.multiarray.array\DUrole{p}{,}\DUrole{w}{  }bool\DUrole{p}{{]}}\DUrole{p}{{]}}}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{{[}{]}}}, \emph{\DUrole{n}{if\_restart}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{Optional\DUrole{p}{{[}}bool\DUrole{p}{{]}}}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{None}}, \emph{\DUrole{n}{restart\_filename}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{str}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{\textquotesingle{}opt\_state.dat\textquotesingle{}}}}{}
\sphinxAtStartPar
Bases: \sphinxcode{\sphinxupquote{object}}

\sphinxAtStartPar
Bayesian optimization class.
\begin{quote}\begin{description}
\item[{Variables}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{curr\_iter}} \textendash{} iteration number of optimizer

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{n\_surrogate\_opt}} \textendash{} number of optima to get during surrogate optimization

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{n\_init}} \textendash{} initial population size (\textgreater{}2)

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{n\_dim}} \textendash{} dimensions of optimization

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{u\_bound}} (\sphinxstyleliteralemphasis{\sphinxupquote{l\_bound}}\sphinxstyleliteralemphasis{\sphinxupquote{,}}) \textendash{} lower and upper bounds on variables

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{total\_population}} \textendash{} total set of population: list of arrays

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{func\_vector}} \textendash{} functional values for the population : scalar for each population

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{K}} \textendash{} covariance matrix

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{K\_inv}} \textendash{} inverse of covariance matrix

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{K\_inv\_y}} \textendash{} K\_inv * func\_vector : pre\sphinxhyphen{}computation to save costly / unstable inversion

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{acquisition\_function}} \textendash{} acquisition function to optimize the surrogate

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{cost\_function}} \textendash{} cost function to MINIMIZE: should be able to take a vector of locations

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{constraints}} \textendash{} list of constraint functions (only points that satisfy these functions will be visited)     NOT enforced in the default init strategy!!

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{if\_restart}} \textendash{} whether it should restart or not

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{restart\_filename}} \textendash{} file to restart from. it will be \sphinxtitleref{opt\_state.dat} if nothing is specified

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{acq\_func\_optimizer}} \textendash{} optimizer for acquisition function. Should take in function, initial guess, bounds and     function derivative. Returns the optimal location

\end{itemize}

\end{description}\end{quote}
\index{acquisition() (PARyOpt.paryopt.BayesOpt method)@\spxentry{acquisition()}\spxextra{PARyOpt.paryopt.BayesOpt method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt:PARyOpt.paryopt.BayesOpt.acquisition}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{acquisition}}}{\emph{\DUrole{n}{x}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{numpy.core.multiarray.array}}, \emph{\DUrole{n}{kappa}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{float}}, \emph{\DUrole{n}{avoid}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{Optional\DUrole{p}{{[}}List\DUrole{p}{{[}}numpy.core.multiarray.array\DUrole{p}{{]}}\DUrole{p}{{]}}}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{None}}, \emph{\DUrole{n}{opt\_indx}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{int}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{0}}}{{ $\rightarrow$ float}}
\sphinxAtStartPar
Calculates the acquisition function + penalty function at the given query location.
If the query point is outside the bounds, it will return an infinity
This is the function that needs to be optimized.
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{x}} \textendash{} location to evaluate acquisition function : shape: (1, self.n\_dim)

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{kappa}} \textendash{} kappa value to pass to acquisition function

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{avoid}} \textendash{} list of x values to avoid (forwarded to penalty function)

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{opt\_indx}} \textendash{} index of acquisition function that is being called

\end{itemize}

\item[{Returns}] \leavevmode
\sphinxAtStartPar
scalar acquisition function at x

\end{description}\end{quote}

\end{fulllineitems}

\index{add\_point() (PARyOpt.paryopt.BayesOpt method)@\spxentry{add\_point()}\spxextra{PARyOpt.paryopt.BayesOpt method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt:PARyOpt.paryopt.BayesOpt.add_point}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{add\_point}}}{\emph{\DUrole{n}{x}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{numpy.core.multiarray.array}}, \emph{\DUrole{n}{y}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{Optional\DUrole{p}{{[}}float\DUrole{p}{{]}}}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{None}}, \emph{\DUrole{n}{if\_check\_nearness}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{bool}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{True}}, \emph{\DUrole{n}{is\_failed}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{bool}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{False}}}{{ $\rightarrow$ None}}
\sphinxAtStartPar
Adds ONLY ONE point to the current set of data we have.
if\_check\_nearness specifies if it is needed to check nearness constraints before adding
the point into total\_population. This should be true if the user is
manually adding points into the population.
Updates following variables: K, K\_inv, total\_population, func\_vector
If the point is a failed evaluation, then it does not add into total\_population and func\_vector
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{x}} \textendash{} array of shape (1, self.n\_dim)

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{y}} \textendash{} cost function value at that location

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{if\_check\_nearness}} \textendash{} boolean, whether to check nearness or not.

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{is\_failed}} \textendash{} boolean, whether the point to add is a failed evaluation or not

\end{itemize}

\item[{Returns}] \leavevmode
\sphinxAtStartPar
None

\end{description}\end{quote}

\end{fulllineitems}

\index{estimate\_best\_kernel\_parameters() (PARyOpt.paryopt.BayesOpt method)@\spxentry{estimate\_best\_kernel\_parameters()}\spxextra{PARyOpt.paryopt.BayesOpt method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt:PARyOpt.paryopt.BayesOpt.estimate_best_kernel_parameters}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{estimate\_best\_kernel\_parameters}}}{\emph{\DUrole{n}{theta\_bounds}\DUrole{o}{=}\DUrole{default_value}{None}}}{{ $\rightarrow$ None}}
\sphinxAtStartPar
Calculates and \sphinxstyleemphasis{sets} the best shape\sphinxhyphen{}parameter/characteristic length\sphinxhyphen{}scale for RBF kernel function and
applies it to the current model.
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{theta\_bounds}} \textendash{} array of bounds for theta

\item[{Returns}] \leavevmode
\sphinxAtStartPar
None

\end{description}\end{quote}

\end{fulllineitems}

\index{evaluate\_surrogate\_at() (PARyOpt.paryopt.BayesOpt method)@\spxentry{evaluate\_surrogate\_at()}\spxextra{PARyOpt.paryopt.BayesOpt method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt:PARyOpt.paryopt.BayesOpt.evaluate_surrogate_at}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{evaluate\_surrogate\_at}}}{\emph{\DUrole{n}{x}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{numpy.core.multiarray.array}}, \emph{\DUrole{n}{include\_failed}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{bool}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{False}}}{{ $\rightarrow$ tuple}}
\sphinxAtStartPar
Evaluates the surrogate at given point
Taken from : \sphinxurl{https://arxiv.org/pdf/1012.2599.pdf} :
“A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to
Active User Modeling and Hierarchical Reinforcement Learning”
Note that it returns sigma\textasciicircum{}2 and not sigma

\sphinxAtStartPar
The mean of the surrogate will not be affected by the failed points, they will only affect
the variance of the surrogate
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{x}} \textendash{} location to evaluate the surrogate rbf approximation

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{include\_failed}} \textendash{} whether to include failed points for surrogate

\end{itemize}

\item[{Returns}] \leavevmode
\sphinxAtStartPar
mean(mu), variance (sigma\textasciicircum{}2)

\end{description}\end{quote}

\end{fulllineitems}

\index{export\_csv() (PARyOpt.paryopt.BayesOpt method)@\spxentry{export\_csv()}\spxextra{PARyOpt.paryopt.BayesOpt method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt:PARyOpt.paryopt.BayesOpt.export_csv}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{export\_csv}}}{\emph{\DUrole{n}{path}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{str}}, \emph{\DUrole{o}{**}\DUrole{n}{kwargs}}}{{ $\rightarrow$ None}}
\sphinxAtStartPar
Writes the data from all completed function evaluations (i.e. x, y pairs) to a CSV file.
The file starts with the following header row: x0, x1, …, x{[}n\_dim\sphinxhyphen{}1{]}, y.
Additional arguments will be forwarded to the csv.writer function, which can be used to control the
formatting of the CSV file. To write a TSV instead of CSV, you can pass dialect=’excel\sphinxhyphen{}tab’.
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{path}} \textendash{} Path to write the file to. If the file already exists, it will be overwritten.

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{kwargs}} \textendash{} Forwarded to csv.writer(), can be used to override the CSV dialect and set formatting options.

\end{itemize}

\end{description}\end{quote}

\end{fulllineitems}

\index{get\_current\_best() (PARyOpt.paryopt.BayesOpt method)@\spxentry{get\_current\_best()}\spxextra{PARyOpt.paryopt.BayesOpt method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt:PARyOpt.paryopt.BayesOpt.get_current_best}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{get\_current\_best}}}{}{{ $\rightarrow$ tuple}}
\sphinxAtStartPar
returns the location of current best and value of current best
\begin{quote}\begin{description}
\item[{Returns}] \leavevmode
\sphinxAtStartPar
location, func\_value

\end{description}\end{quote}

\end{fulllineitems}

\index{get\_current\_iteration() (PARyOpt.paryopt.BayesOpt method)@\spxentry{get\_current\_iteration()}\spxextra{PARyOpt.paryopt.BayesOpt method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt:PARyOpt.paryopt.BayesOpt.get_current_iteration}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{get\_current\_iteration}}}{}{{ $\rightarrow$ int}}
\sphinxAtStartPar
Returns the current iteration number
\begin{quote}\begin{description}
\item[{Returns}] \leavevmode
\sphinxAtStartPar
integer

\end{description}\end{quote}

\end{fulllineitems}

\index{get\_total\_population() (PARyOpt.paryopt.BayesOpt method)@\spxentry{get\_total\_population()}\spxextra{PARyOpt.paryopt.BayesOpt method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt:PARyOpt.paryopt.BayesOpt.get_total_population}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{get\_total\_population}}}{}{{ $\rightarrow$ Tuple\DUrole{p}{{[}}List\DUrole{p}{{[}}numpy.core.multiarray.array\DUrole{p}{{]}}\DUrole{p}{,}\DUrole{w}{  }List\DUrole{p}{{[}}float\DUrole{p}{{]}}\DUrole{p}{{]}}}}
\sphinxAtStartPar
returns the total population : for visualization purposes
\begin{quote}\begin{description}
\item[{Returns}] \leavevmode
\sphinxAtStartPar
total\_population and function values at that location

\end{description}\end{quote}

\end{fulllineitems}

\index{penalty\_function() (PARyOpt.paryopt.BayesOpt method)@\spxentry{penalty\_function()}\spxextra{PARyOpt.paryopt.BayesOpt method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt:PARyOpt.paryopt.BayesOpt.penalty_function}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{penalty\_function}}}{\emph{\DUrole{n}{x}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{numpy.core.multiarray.array}}, \emph{\DUrole{n}{kappa}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{float}}, \emph{\DUrole{n}{avoid}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{List\DUrole{p}{{[}}numpy.core.multiarray.array\DUrole{p}{{]}}}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{{[}{]}}}}{{ $\rightarrow$ float}}
\sphinxAtStartPar
Calculates the penalty function, to prevent local optima to repeat
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{x}} \textendash{} location

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{kappa}} \textendash{} kappa value used in acquisition function

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{avoid}} \textendash{} extra points to avoid (other than self.total\_pop/self.pending/self.failed)

\end{itemize}

\item[{Returns}] \leavevmode
\sphinxAtStartPar
penalty function value at x

\end{description}\end{quote}

\end{fulllineitems}

\index{set\_new\_kernel\_parameters() (PARyOpt.paryopt.BayesOpt method)@\spxentry{set\_new\_kernel\_parameters()}\spxextra{PARyOpt.paryopt.BayesOpt method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt:PARyOpt.paryopt.BayesOpt.set_new_kernel_parameters}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{set\_new\_kernel\_parameters}}}{\emph{\DUrole{n}{theta}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{Union\DUrole{p}{{[}}float\DUrole{p}{,}\DUrole{w}{  }numpy.core.multiarray.array\DUrole{p}{{]}}}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{1.0}}, \emph{\DUrole{n}{theta0}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{float}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{1.0}}}{{ $\rightarrow$ None}}
\sphinxAtStartPar
sets the kernel parameters: called from the user script.
The covariance matrix needs to be re\sphinxhyphen{}calculated when this function is called
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{theta}} \textendash{} scaling of dimensions used with the distance function

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{theta0}} \textendash{} scaling of kernel function

\end{itemize}

\item[{Returns}] \leavevmode
\sphinxAtStartPar
None

\end{description}\end{quote}

\end{fulllineitems}

\index{update\_iter() (PARyOpt.paryopt.BayesOpt method)@\spxentry{update\_iter()}\spxextra{PARyOpt.paryopt.BayesOpt method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt:PARyOpt.paryopt.BayesOpt.update_iter}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{update\_iter}}}{\emph{\DUrole{n}{iter\_max}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{int}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{1}}}{{ $\rightarrow$ None}}
\sphinxAtStartPar
Finds the next query point and updates the data for the next iteration
\begin{quote}\begin{description}
\item[{Returns}] \leavevmode
\sphinxAtStartPar
None

\end{description}\end{quote}

\end{fulllineitems}

\index{update\_surrogate() (PARyOpt.paryopt.BayesOpt method)@\spxentry{update\_surrogate()}\spxextra{PARyOpt.paryopt.BayesOpt method}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt:PARyOpt.paryopt.BayesOpt.update_surrogate}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{update\_surrogate}}}{}{{ $\rightarrow$ None}}
\sphinxAtStartPar
updates the inverse of covariance function.
In order to incorporate failed locations as interpolated values, this update is 2 staged:
\begin{enumerate}
\sphinxsetlistlabels{\arabic}{enumi}{enumii}{}{.}%
\item {} 
\sphinxAtStartPar
Create the surrogate (K,K\_inv) with only the successfully evaluated points

\item {} \begin{enumerate}
\sphinxsetlistlabels{\alph}{enumii}{enumiii}{}{)}%
\item {} 
\sphinxAtStartPar
Evaluate the mean of the surrogate at the failed points

\item {} 
\sphinxAtStartPar
Incorporate the interpolated failed locations into the surrogate and update the surrogate (K,K\_inv)

\end{enumerate}

\end{enumerate}

\sphinxAtStartPar
Updates the values of self.K\_success, self.K\_inv\_success, self.K\_inv\_y\_success, self.K\_inv and self.K\_inv\_y
\begin{quote}\begin{description}
\item[{Returns}] \leavevmode
\sphinxAtStartPar
None

\end{description}\end{quote}

\end{fulllineitems}


\end{fulllineitems}



\section{PARyOpt.utils module}
\label{\detokenize{PARyOpt:module-PARyOpt.utils}}\label{\detokenize{PARyOpt:paryopt-utils-module}}\index{module@\spxentry{module}!PARyOpt.utils@\spxentry{PARyOpt.utils}}\index{PARyOpt.utils@\spxentry{PARyOpt.utils}!module@\spxentry{module}}
\sphinxAtStartPar
Utility functions that will be used in kernels, acquisition functions
and bayes optimization
\index{cdf\_normal() (in module PARyOpt.utils)@\spxentry{cdf\_normal()}\spxextra{in module PARyOpt.utils}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt:PARyOpt.utils.cdf_normal}}\pysiglinewithargsret{\sphinxcode{\sphinxupquote{PARyOpt.utils.}}\sphinxbfcode{\sphinxupquote{cdf\_normal}}}{\emph{\DUrole{n}{x}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{numpy.core.multiarray.array}}, \emph{\DUrole{n}{mean}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{numpy.core.multiarray.array}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{0.0}}, \emph{\DUrole{n}{sigma\_sq}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{numpy.core.multiarray.array}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{1.0}}}{{ $\rightarrow$ float}}
\sphinxAtStartPar
Cumulative distribution function of standard normal distribution
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{x}} \textendash{} scalar / location

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{mean}} \textendash{} mean of distribution

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{sigma\_sq}} \textendash{} variance of distribution (sigma\textasciicircum{}2)

\end{itemize}

\item[{Returns}] \leavevmode
\sphinxAtStartPar
cdf of normal distribution

\end{description}\end{quote}

\end{fulllineitems}

\index{distance() (in module PARyOpt.utils)@\spxentry{distance()}\spxextra{in module PARyOpt.utils}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt:PARyOpt.utils.distance}}\pysiglinewithargsret{\sphinxcode{\sphinxupquote{PARyOpt.utils.}}\sphinxbfcode{\sphinxupquote{distance}}}{\emph{\DUrole{n}{x1}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{numpy.core.multiarray.array}}, \emph{\DUrole{n}{x2}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{numpy.core.multiarray.array}}}{{ $\rightarrow$ float}}
\sphinxAtStartPar
returns the distance between two query points
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{x1}} \textendash{} point 1

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{x2}} \textendash{} point 2

\end{itemize}

\item[{Returns}] \leavevmode
\sphinxAtStartPar
euclidean distance between the two points

\end{description}\end{quote}

\end{fulllineitems}

\index{erf() (in module PARyOpt.utils)@\spxentry{erf()}\spxextra{in module PARyOpt.utils}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt:PARyOpt.utils.erf}}\pysiglinewithargsret{\sphinxcode{\sphinxupquote{PARyOpt.utils.}}\sphinxbfcode{\sphinxupquote{erf}}}{\emph{\DUrole{n}{x}}}{}
\sphinxAtStartPar
error function of x: used in calculating cumulative distribution.
Unable to import from scipy, so writing our own function
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{x}} \textendash{} float

\item[{Returns}] \leavevmode
\sphinxAtStartPar
error function of x

\end{description}\end{quote}

\end{fulllineitems}

\index{lhs() (in module PARyOpt.utils)@\spxentry{lhs()}\spxextra{in module PARyOpt.utils}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt:PARyOpt.utils.lhs}}\pysiglinewithargsret{\sphinxcode{\sphinxupquote{PARyOpt.utils.}}\sphinxbfcode{\sphinxupquote{lhs}}}{\emph{\DUrole{n}{n}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{int}}, \emph{\DUrole{n}{samples}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{Optional\DUrole{p}{{[}}int\DUrole{p}{{]}}}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{None}}, \emph{\DUrole{n}{criterion}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{Optional\DUrole{p}{{[}}str\DUrole{p}{{]}}}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{None}}, \emph{\DUrole{n}{iterations}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{Optional\DUrole{p}{{[}}int\DUrole{p}{{]}}}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{None}}}{}
\sphinxAtStartPar
Generate a latin\sphinxhyphen{}hypercube design
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{n}} \textendash{} The number of factors to generate samples for

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{samples}} \textendash{} The number of samples to generate for each factor (Default: n)

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{criterion}} \textendash{} Allowable values are “center” or “c”, “maximin” or “m”, “centermaximin” or “cm”, and     “correlation” or “corr”. If no value given, the design is centermaximin.

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{iterations}} \textendash{} The number of iterations in the maximin and correlations algorithms (Default: 5).

\end{itemize}

\item[{Return H}] \leavevmode
\sphinxAtStartPar
An n\sphinxhyphen{}by\sphinxhyphen{}samples design matrix that has been normalized so factor values are uniformly      spaced between zero and one.

\item[{Example}] \leavevmode
\end{description}\end{quote}

\sphinxAtStartPar
A 3\sphinxhyphen{}factor design (defaults to 3 samples):

\begin{sphinxVerbatim}[commandchars=\\\{\}]
\PYG{g+gp}{\PYGZgt{}\PYGZgt{}\PYGZgt{} }\PYG{n}{lhs}\PYG{p}{(}\PYG{l+m+mi}{3}\PYG{p}{)}
\PYG{g+go}{array([[ 0.40069325,  0.08118402,  0.69763298],}
\PYG{g+go}{       [ 0.19524568,  0.41383587,  0.29947106],}
\PYG{g+go}{       [ 0.85341601,  0.75460699,  0.360024  ]])}
\end{sphinxVerbatim}

\sphinxAtStartPar
A 4\sphinxhyphen{}factor design with 6 samples:

\begin{sphinxVerbatim}[commandchars=\\\{\}]
\PYG{g+gp}{\PYGZgt{}\PYGZgt{}\PYGZgt{} }\PYG{n}{lhs}\PYG{p}{(}\PYG{l+m+mi}{4}\PYG{p}{,} \PYG{n}{samples}\PYG{o}{=}\PYG{l+m+mi}{6}\PYG{p}{)}
\PYG{g+go}{array([[ 0.27226812,  0.02811327,  0.62792445,  0.91988196],}
\PYG{g+go}{       [ 0.76945538,  0.43501682,  0.01107457,  0.09583358],}
\PYG{g+go}{       [ 0.45702981,  0.76073773,  0.90245401,  0.18773015],}
\PYG{g+go}{       [ 0.99342115,  0.85814198,  0.16996665,  0.65069309],}
\PYG{g+go}{       [ 0.63092013,  0.22148567,  0.33616859,  0.36332478],}
\PYG{g+go}{       [ 0.05276917,  0.5819198 ,  0.67194243,  0.78703262]])}
\end{sphinxVerbatim}

\sphinxAtStartPar
A 2\sphinxhyphen{}factor design with 5 centered samples:

\begin{sphinxVerbatim}[commandchars=\\\{\}]
\PYG{g+gp}{\PYGZgt{}\PYGZgt{}\PYGZgt{} }\PYG{n}{lhs}\PYG{p}{(}\PYG{l+m+mi}{2}\PYG{p}{,} \PYG{n}{samples}\PYG{o}{=}\PYG{l+m+mi}{5}\PYG{p}{,} \PYG{n}{criterion}\PYG{o}{=}\PYG{l+s+s1}{\PYGZsq{}}\PYG{l+s+s1}{center}\PYG{l+s+s1}{\PYGZsq{}}\PYG{p}{)}
\PYG{g+go}{array([[ 0.3,  0.5],}
\PYG{g+go}{       [ 0.7,  0.9],}
\PYG{g+go}{       [ 0.1,  0.3],}
\PYG{g+go}{       [ 0.9,  0.1],}
\PYG{g+go}{       [ 0.5,  0.7]])}
\end{sphinxVerbatim}

\sphinxAtStartPar
A 3\sphinxhyphen{}factor design with 4 samples where the minimum distance between
all samples has been maximized:

\begin{sphinxVerbatim}[commandchars=\\\{\}]
\PYG{g+gp}{\PYGZgt{}\PYGZgt{}\PYGZgt{} }\PYG{n}{lhs}\PYG{p}{(}\PYG{l+m+mi}{3}\PYG{p}{,} \PYG{n}{samples}\PYG{o}{=}\PYG{l+m+mi}{4}\PYG{p}{,} \PYG{n}{criterion}\PYG{o}{=}\PYG{l+s+s1}{\PYGZsq{}}\PYG{l+s+s1}{maximin}\PYG{l+s+s1}{\PYGZsq{}}\PYG{p}{)}
\PYG{g+go}{array([[ 0.02642564,  0.55576963,  0.50261649],}
\PYG{g+go}{       [ 0.51606589,  0.88933259,  0.34040838],}
\PYG{g+go}{       [ 0.98431735,  0.0380364 ,  0.01621717],}
\PYG{g+go}{       [ 0.40414671,  0.33339132,  0.84845707]])}
\end{sphinxVerbatim}

\sphinxAtStartPar
A 4\sphinxhyphen{}factor design with 5 samples where the samples are as uncorrelated
as possible (within 10 iterations):

\begin{sphinxVerbatim}[commandchars=\\\{\}]
\PYG{g+gp}{\PYGZgt{}\PYGZgt{}\PYGZgt{} }\PYG{n}{lhs}\PYG{p}{(}\PYG{l+m+mi}{4}\PYG{p}{,} \PYG{n}{samples}\PYG{o}{=}\PYG{l+m+mi}{5}\PYG{p}{,} \PYG{n}{criterion}\PYG{o}{=}\PYG{l+s+s1}{\PYGZsq{}}\PYG{l+s+s1}{correlate}\PYG{l+s+s1}{\PYGZsq{}}\PYG{p}{,} \PYG{n}{iterations}\PYG{o}{=}\PYG{l+m+mi}{10}\PYG{p}{)}
\end{sphinxVerbatim}

\end{fulllineitems}

\index{pdf\_normal() (in module PARyOpt.utils)@\spxentry{pdf\_normal()}\spxextra{in module PARyOpt.utils}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{PARyOpt:PARyOpt.utils.pdf_normal}}\pysiglinewithargsret{\sphinxcode{\sphinxupquote{PARyOpt.utils.}}\sphinxbfcode{\sphinxupquote{pdf\_normal}}}{\emph{\DUrole{n}{x}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{numpy.core.multiarray.array}}, \emph{\DUrole{n}{mean}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{numpy.core.multiarray.array}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{0.0}}, \emph{\DUrole{n}{sigma\_sq}\DUrole{p}{:}\DUrole{w}{  }\DUrole{n}{float}\DUrole{w}{  }\DUrole{o}{=}\DUrole{w}{  }\DUrole{default_value}{1.0}}}{{ $\rightarrow$ float}}
\sphinxAtStartPar
Probability distribution function of standard normal distribution,
returns the pdf of a location from given mean with variance sigma\_sq
\begin{quote}\begin{description}
\item[{Parameters}] \leavevmode\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{x}} \textendash{} scalar/location

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{mean}} \textendash{} mean of distribution

\item {} 
\sphinxAtStartPar
\sphinxstyleliteralstrong{\sphinxupquote{sigma\_sq}} \textendash{} variance of distribution (sigma\textasciicircum{}2)

\end{itemize}

\item[{Returns}] \leavevmode
\sphinxAtStartPar
pdf of normal distribution

\end{description}\end{quote}

\end{fulllineitems}



\section{Module contents}
\label{\detokenize{PARyOpt:module-PARyOpt}}\label{\detokenize{PARyOpt:module-contents}}\index{module@\spxentry{module}!PARyOpt@\spxentry{PARyOpt}}\index{PARyOpt@\spxentry{PARyOpt}!module@\spxentry{module}}

\chapter{docs}
\label{\detokenize{modules:docs}}\label{\detokenize{modules::doc}}

\section{conf module}
\label{\detokenize{conf:module-conf}}\label{\detokenize{conf:conf-module}}\label{\detokenize{conf::doc}}\index{module@\spxentry{module}!conf@\spxentry{conf}}\index{conf@\spxentry{conf}!module@\spxentry{module}}\index{Mock (class in conf)@\spxentry{Mock}\spxextra{class in conf}}

\begin{fulllineitems}
\phantomsection\label{\detokenize{conf:conf.Mock}}\pysiglinewithargsret{\sphinxbfcode{\sphinxupquote{class\DUrole{w}{  }}}\sphinxcode{\sphinxupquote{conf.}}\sphinxbfcode{\sphinxupquote{Mock}}}{\emph{\DUrole{o}{*}\DUrole{n}{args}}, \emph{\DUrole{o}{**}\DUrole{n}{kw}}}{}
\sphinxAtStartPar
Bases: \sphinxcode{\sphinxupquote{unittest.mock.MagicMock}}

\end{fulllineitems}



\chapter{Examples}
\label{\detokenize{examples/index:examples}}\label{\detokenize{examples/index::doc}}
\sphinxAtStartPar
This provides a demonstration of the exhaustive functionality of PARyOpt. These are very intricately connected to
the examples on BitBucket, so the user is suggested to go through them simultaneously. The examples are structured as
follows:


\begin{savenotes}\sphinxattablestart
\raggedleft
\sphinxcapstartof{table}
\sphinxthecaptionisattop
\sphinxcaption{Description of Examples}\label{\detokenize{examples/index:id1}}
\sphinxaftertopcaption
\begin{tabular}[t]{|\X{15}{45}|\X{30}{45}|}
\hline
\sphinxstyletheadfamily 
\sphinxAtStartPar
Example \#
&\sphinxstyletheadfamily 
\sphinxAtStartPar
Description
\\
\hline
\sphinxAtStartPar
Example \sphinxstylestrong{0}

\sphinxAtStartPar
{\hyperref[\detokenize{examples/getting_started_example:example-0}]{\sphinxcrossref{\DUrole{std,std-ref}{Getting Started}}}}
&
\sphinxAtStartPar
A dive into setting up the optimization routine.
\\
\hline
\sphinxAtStartPar
Example \sphinxstylestrong{1}

\sphinxAtStartPar
{\hyperref[\detokenize{examples/user_defined_function_example:example-1}]{\sphinxcrossref{\DUrole{std,std-ref}{Custom functions for surrogate construction}}}}
&
\sphinxAtStartPar
Using the same problem as example 0, the modularity of the
framework is demonstrated. All the functions that can be customized are shown.
\\
\hline
\sphinxAtStartPar
Example \sphinxstylestrong{2}

\sphinxAtStartPar
{\hyperref[\detokenize{examples/restart_example:example-2}]{\sphinxcrossref{\DUrole{std,std-ref}{Restart from previous state}}}}
&
\sphinxAtStartPar
Explains the restart capabilities of the framework and how
one can use in case of (hardware/resource) failure.
\\
\hline
\sphinxAtStartPar
Example \sphinxstylestrong{3}

\sphinxAtStartPar
{\hyperref[\detokenize{examples/async_local_evaluator_example:example-3}]{\sphinxcrossref{\DUrole{std,std-ref}{Local asynchronous evaluator}}}}
&
\sphinxAtStartPar
Asynchronocity is introduced and a local asynchronous implementation
of example 0 is shown. This implementation can be easily extended to an asynchronous
remote evaluator.
\\
\hline
\sphinxAtStartPar
Example \sphinxstylestrong{4}

\sphinxAtStartPar
{\hyperref[\detokenize{examples/kriging_example:example-4}]{\sphinxcrossref{\DUrole{std,std-ref}{Kriging}}}}
&
\sphinxAtStartPar
Explains the kriging functionality of the framework and how data
can be assimilated to perform kriging
\\
\hline
\end{tabular}
\par
\sphinxattableend\end{savenotes}


\section{Getting Started}
\label{\detokenize{examples/getting_started_example:getting-started}}\label{\detokenize{examples/getting_started_example:example-0}}\label{\detokenize{examples/getting_started_example::doc}}
\sphinxAtStartPar
This example shows how one gets started with the optimization software.
One is expected to see through \sphinxstylestrong{example\_0.py} for a better understanding.

\sphinxAtStartPar
Just like any optimization, we should have information about the following:
\begin{enumerate}
\sphinxsetlistlabels{\arabic}{enumi}{enumii}{}{.}%
\item {} 
\sphinxAtStartPar
dimensionality of the problem \textendash{} \sphinxcode{\sphinxupquote{n\_dim}}

\item {} 
\sphinxAtStartPar
cost function that has to be \sphinxstylestrong{minimized} \textendash{} \sphinxcode{\sphinxupquote{function}}

\item {} 
\sphinxAtStartPar
bounds on the variables   \textendash{} \sphinxcode{\sphinxupquote{l\_bound}}, \sphinxcode{\sphinxupquote{u\_bound}}

\end{enumerate}

\sphinxAtStartPar
Some more parameters for Bayesian Optimization
\begin{enumerate}
\sphinxsetlistlabels{\arabic}{enumi}{enumii}{}{.}%
\setcounter{enumi}{3}
\item {} 
\sphinxAtStartPar
number of initial evaluations for constructing the prior \textendash{}\sphinxcode{\sphinxupquote{n\_init}}

\item {} 
\sphinxAtStartPar
type of kernel function \textendash{} \sphinxcode{\sphinxupquote{kern\_function}}

\item {} 
\sphinxAtStartPar
number of evaluations per iteration \textendash{} \sphinxcode{\sphinxupquote{n\_opt}}

\item {} 
\sphinxAtStartPar
platform of evaluation \sphinxhyphen{} local computer / remote computer

\item {} 
\sphinxAtStartPar
parallel / serial evaluations \textendash{} \sphinxcode{\sphinxupquote{cost\_function}}

\item {} 
\sphinxAtStartPar
asynchronocity of evaluations \textendash{} \sphinxcode{\sphinxupquote{cost\_function}}

\item {} 
\sphinxAtStartPar
acquisition function (list) \textendash{} \sphinxcode{\sphinxupquote{acq\_func}}
\begin{itemize}
\item {} 
\sphinxAtStartPar
parallelization of acquisition function \textendash{} \sphinxcode{\sphinxupquote{kappa\_strategy}}

\end{itemize}

\end{enumerate}

\sphinxAtStartPar
The \sphinxcode{\sphinxupquote{kappa\_strategy}} defines exploration vs exploitation of the optimizer. Those with a \sphinxstyleemphasis{large} \sphinxcode{\sphinxupquote{kappa}} value
will explore and a \sphinxstyleemphasis{small} \sphinxcode{\sphinxupquote{kappa}} value will exploit.

\sphinxAtStartPar
In this example, we shall solve a simple parabolic cost function, on a local machine with no parallelization. The evaluations
will, therefore, be fully synchronous. As part of the example, we shall do an exploration dominated search in the
optimization. The parameters that will be used are as follows:

\begin{sphinxVerbatim}[commandchars=\\\{\}]
\PYG{k+kn}{from} \PYG{n+nn}{PARyOpt}\PYG{n+nn}{.}\PYG{n+nn}{evaluators} \PYG{k+kn}{import} \PYG{n}{FunctionEvaluator}
\PYG{k+kn}{import} \PYG{n+nn}{numpy} \PYG{k}{as} \PYG{n+nn}{np}
\PYG{k+kn}{from} \PYG{n+nn}{PARyOpt} \PYG{k+kn}{import} \PYG{n}{BayesOpt}

\PYG{n}{n\PYGZus{}dim} \PYG{o}{=} \PYG{l+m+mi}{1}
\PYG{n}{l\PYGZus{}bound} \PYG{o}{=} \PYG{n}{np}\PYG{o}{.}\PYG{n}{asarray}\PYG{p}{(}\PYG{p}{[}\PYG{o}{\PYGZhy{}}\PYG{l+m+mf}{12.}\PYG{p}{]}\PYG{p}{)}
\PYG{n}{u\PYGZus{}bound} \PYG{o}{=} \PYG{n}{np}\PYG{o}{.}\PYG{n}{asarray}\PYG{p}{(}\PYG{p}{[}\PYG{l+m+mf}{12.}\PYG{p}{]}\PYG{p}{)}

\PYG{n}{n\PYGZus{}init} \PYG{o}{=} \PYG{l+m+mi}{2}
\PYG{n}{kern\PYGZus{}function} \PYG{o}{=} \PYG{l+s+s1}{\PYGZsq{}}\PYG{l+s+s1}{sqr\PYGZus{}exp}\PYG{l+s+s1}{\PYGZsq{}}       \PYG{c+c1}{\PYGZsh{} squared exponential}
\PYG{n}{acq\PYGZus{}func} \PYG{o}{=} \PYG{l+s+s1}{\PYGZsq{}}\PYG{l+s+s1}{LCB}\PYG{l+s+s1}{\PYGZsq{}}                \PYG{c+c1}{\PYGZsh{} lower confidence bound}
\PYG{k}{def} \PYG{n+nf}{my\PYGZus{}cost\PYGZus{}function}\PYG{p}{(}\PYG{n}{x}\PYG{p}{:} \PYG{n}{np}\PYG{o}{.}\PYG{n}{array}\PYG{p}{)} \PYG{o}{\PYGZhy{}}\PYG{o}{\PYGZgt{}} \PYG{n+nb}{float}\PYG{p}{:}
        \PYG{n}{y} \PYG{o}{=} \PYG{n}{np}\PYG{o}{.}\PYG{n}{sum}\PYG{p}{(}\PYG{p}{(}\PYG{n}{x}\PYG{o}{\PYGZhy{}}\PYG{l+m+mf}{2.5}\PYG{p}{)} \PYG{o}{*}\PYG{o}{*} \PYG{l+m+mi}{2} \PYG{o}{+} \PYG{l+m+mi}{5}\PYG{p}{)}
        \PYG{k}{return} \PYG{n+nb}{float}\PYG{p}{(}\PYG{n}{y}\PYG{p}{)}
\PYG{c+c1}{\PYGZsh{} instantiate an evaluator that evaluates serially on the local machine}
\PYG{n}{evaluator} \PYG{o}{=} \PYG{n}{FunctionEvaluator}\PYG{p}{(}\PYG{n}{my\PYGZus{}cost\PYGZus{}function}\PYG{p}{)}
\PYG{k}{def} \PYG{n+nf}{my\PYGZus{}kappa}\PYG{p}{(}\PYG{n}{curr\PYGZus{}iter}\PYG{p}{:} \PYG{n+nb}{int}\PYG{p}{)} \PYG{o}{\PYGZhy{}}\PYG{o}{\PYGZgt{}} \PYG{n+nb}{float}\PYG{p}{:}
    \PYG{k}{return} \PYG{l+m+mf}{1000.0}           \PYG{c+c1}{\PYGZsh{} large value for exploration}
\end{sphinxVerbatim}


\subsection{Initialization}
\label{\detokenize{examples/getting_started_example:initialization}}
\sphinxAtStartPar
Having defined these parameters, we shall now initialize the optimizer:

\begin{sphinxVerbatim}[commandchars=\\\{\}]
\PYG{n}{b\PYGZus{}opt} \PYG{o}{=} \PYG{n}{BayesOpt}\PYG{p}{(}\PYG{n}{cost\PYGZus{}function}\PYG{o}{=}\PYG{n}{evaluator}\PYG{p}{,}
                 \PYG{n}{l\PYGZus{}bound}\PYG{o}{=}\PYG{n}{l\PYGZus{}bound}\PYG{p}{,} \PYG{n}{u\PYGZus{}bound}\PYG{o}{=}\PYG{n}{u\PYGZus{}bound}\PYG{p}{,} \PYG{n}{n\PYGZus{}dim}\PYG{o}{=}\PYG{n}{n\PYGZus{}dim}\PYG{p}{,}
                 \PYG{n}{n\PYGZus{}init}\PYG{o}{=}\PYG{l+m+mi}{2}\PYG{p}{,}
                 \PYG{n}{kern\PYGZus{}function}\PYG{o}{=}\PYG{l+s+s1}{\PYGZsq{}}\PYG{l+s+s1}{sqr\PYGZus{}exp}\PYG{l+s+s1}{\PYGZsq{}}\PYG{p}{,}
                 \PYG{n}{acq\PYGZus{}func}\PYG{o}{=}\PYG{l+s+s1}{\PYGZsq{}}\PYG{l+s+s1}{LCB}\PYG{l+s+s1}{\PYGZsq{}}\PYG{p}{,}
                 \PYG{n}{n\PYGZus{}opt}\PYG{o}{=}\PYG{l+m+mi}{1}\PYG{p}{,}                           \PYG{c+c1}{\PYGZsh{} default setting}
                 \PYG{n}{kappa\PYGZus{}strategy}\PYG{o}{=}\PYG{n}{my\PYGZus{}kappa}\PYG{p}{,}
                 \PYG{n}{if\PYGZus{}restart}\PYG{o}{=}\PYG{k+kc}{False}\PYG{p}{)}
\end{sphinxVerbatim}

\sphinxAtStartPar
The stage is now set for optimization to be performed. Since this package does not provide any standard termination criteria,
the user is expected to design a termination based on the nature of the problem. In this example, we shall look at
a very simple termination criterion of number of iterations.

\begin{sphinxVerbatim}[commandchars=\\\{\}]
\PYG{n}{max\PYGZus{}iter} \PYG{o}{=} \PYG{l+m+mi}{10}
\end{sphinxVerbatim}


\subsection{Update}
\label{\detokenize{examples/getting_started_example:update}}
\sphinxAtStartPar
The user shall manually update the optimization every iteration. This provides ways to post\sphinxhyphen{}process user required metrics
every iteration, as well as do a regular \sphinxstyleemphasis{hyper\sphinxhyphen{}parameter optimization} for optimized surrogate.

\begin{sphinxVerbatim}[commandchars=\\\{\}]
\PYG{k}{for} \PYG{n}{curr\PYGZus{}iter} \PYG{o+ow}{in} \PYG{n+nb}{range}\PYG{p}{(}\PYG{n}{max\PYGZus{}iter}\PYG{p}{)}\PYG{p}{:}
    \PYG{n}{b\PYGZus{}opt}\PYG{o}{.}\PYG{n}{update\PYGZus{}iter}\PYG{p}{(}\PYG{p}{)}
\end{sphinxVerbatim}


\subsection{Hyper parameter optimization}
\label{\detokenize{examples/getting_started_example:hyper-parameter-optimization}}
\sphinxAtStartPar
An implementation of the standard hyper parameter optimization is done in \sphinxcode{\sphinxupquote{estimate\_best\_kernel\_parameters()}}.
This minimizes a maximum likelihood estimate of the constructed surrogate and eventually sets the \sphinxstyleemphasis{optimal} kernel
length scale. It can be invoked by calling:

\begin{sphinxVerbatim}[commandchars=\\\{\}]
\PYG{n}{theta\PYGZus{}min} \PYG{o}{=} \PYG{l+m+mf}{0.01}
\PYG{n}{theta\PYGZus{}max} \PYG{o}{=} \PYG{l+m+mf}{50.}
\PYG{n}{b\PYGZus{}opt}\PYG{o}{.}\PYG{n}{estimate\PYGZus{}best\PYGZus{}kernel\PYGZus{}parameters}\PYG{p}{(}\PYG{n}{theta\PYGZus{}bounds}\PYG{o}{=}\PYG{p}{[}\PYG{p}{[}\PYG{n}{theta\PYGZus{}min}\PYG{p}{,} \PYG{n}{theta\PYGZus{}max}\PYG{p}{]}\PYG{p}{]}\PYG{p}{)}
\end{sphinxVerbatim}

\sphinxAtStartPar
Hyper parameter optimization need not be performed every iteration as the surrogate may not change much with the
addition of a single data point. Hence its call can be periodic based on the iteration number.


\subsection{Surrogate query}
\label{\detokenize{examples/getting_started_example:surrogate-query}}
\sphinxAtStartPar
Having constructed the surrogate, one may need to query it for several purposes, including visualization, post\sphinxhyphen{}processing
and termination criteria. This functionality is provided through the \sphinxcode{\sphinxupquote{evaluate\_surrogate\_at()}} function. It
returns the value of the mean and variance of the surrogate at the queried location.

\begin{sphinxVerbatim}[commandchars=\\\{\}]
\PYG{n}{location\PYGZus{}to\PYGZus{}query} \PYG{o}{=} \PYG{n}{np}\PYG{o}{.}\PYG{n}{asarray}\PYG{p}{(}\PYG{p}{[}\PYG{l+m+mf}{0.5}\PYG{p}{]}\PYG{p}{)}
\PYG{n}{mean}\PYG{p}{,} \PYG{n}{variance} \PYG{o}{=} \PYG{n}{b\PYGZus{}opt}\PYG{o}{.}\PYG{n}{evaluate\PYGZus{}surrogate\PYGZus{}at}\PYG{p}{(}\PYG{n}{location\PYGZus{}to\PYGZus{}query}\PYG{p}{)}
\end{sphinxVerbatim}


\subsection{Logging}
\label{\detokenize{examples/getting_started_example:logging}}
\sphinxAtStartPar
PARyOpt uses the python \sphinxhref{https://docs.python.org/3/library/logging.html}{logging} module for logging. The user has to
instantiate the logger in the main code. If the logger is not initiated, the logs will be streamed to \sphinxcode{\sphinxupquote{stdout}}. An
example of using the logger is also in the above example:

\begin{sphinxVerbatim}[commandchars=\\\{\}]
\PYG{k+kn}{import} \PYG{n+nn}{logging}\PYG{o}{,} \PYG{n+nn}{time}

\PYG{n}{logger} \PYG{o}{=} \PYG{n}{logging}\PYG{o}{.}\PYG{n}{getLogger}\PYG{p}{(}\PYG{p}{)}
\PYG{n}{logger}\PYG{o}{.}\PYG{n}{setLevel}\PYG{p}{(}\PYG{n}{logging}\PYG{o}{.}\PYG{n}{INFO}\PYG{p}{)}  \PYG{c+c1}{\PYGZsh{} either NOTSET, INFO, DEBUG, WARNING, ERROR, CRITICAL \PYGZhy{}\PYGZhy{} different levels of log}
\PYG{n}{log\PYGZus{}file\PYGZus{}name} \PYG{o}{=} \PYG{l+s+s1}{\PYGZsq{}}\PYG{l+s+s1}{example0\PYGZus{}}\PYG{l+s+si}{\PYGZob{}\PYGZcb{}}\PYG{l+s+s1}{.log}\PYG{l+s+s1}{\PYGZsq{}}\PYG{o}{.}\PYG{n}{format}\PYG{p}{(}\PYG{n}{time}\PYG{o}{.}\PYG{n}{strftime}\PYG{p}{(}\PYG{l+s+s2}{\PYGZdq{}}\PYG{l+s+s2}{\PYGZpc{}}\PYG{l+s+s2}{Y.}\PYG{l+s+s2}{\PYGZpc{}}\PYG{l+s+s2}{m.}\PYG{l+s+si}{\PYGZpc{}d}\PYG{l+s+s2}{\PYGZhy{}}\PYG{l+s+s2}{\PYGZpc{}}\PYG{l+s+s2}{H}\PYG{l+s+s2}{\PYGZpc{}}\PYG{l+s+s2}{M}\PYG{l+s+s2}{\PYGZpc{}}\PYG{l+s+s2}{S}\PYG{l+s+s2}{\PYGZdq{}}\PYG{p}{)}\PYG{p}{)}
\PYG{n}{fh} \PYG{o}{=} \PYG{n}{logging}\PYG{o}{.}\PYG{n}{FileHandler}\PYG{p}{(}\PYG{n}{log\PYGZus{}file\PYGZus{}name}\PYG{p}{,} \PYG{n}{mode}\PYG{o}{=}\PYG{l+s+s1}{\PYGZsq{}}\PYG{l+s+s1}{a}\PYG{l+s+s1}{\PYGZsq{}}\PYG{p}{)}
\PYG{c+c1}{\PYGZsh{} logging format}
\PYG{n}{formatter} \PYG{o}{=} \PYG{n}{logging}\PYG{o}{.}\PYG{n}{Formatter}\PYG{p}{(}\PYG{l+s+s1}{\PYGZsq{}}\PYG{l+s+si}{\PYGZpc{}(asctime)s}\PYG{l+s+s1}{ \PYGZhy{} }\PYG{l+s+si}{\PYGZpc{}(name)s}\PYG{l+s+s1}{ \PYGZhy{} }\PYG{l+s+si}{\PYGZpc{}(levelname)s}\PYG{l+s+s1}{ \PYGZhy{} }\PYG{l+s+si}{\PYGZpc{}(message)s}\PYG{l+s+s1}{\PYGZsq{}}\PYG{p}{)}
\PYG{n}{fh}\PYG{o}{.}\PYG{n}{setFormatter}\PYG{p}{(}\PYG{n}{formatter}\PYG{p}{)}
\PYG{n}{logger}\PYG{o}{.}\PYG{n}{addHandler}\PYG{p}{(}\PYG{n}{fh}\PYG{p}{)}
\end{sphinxVerbatim}


\subsection{Saving data}
\label{\detokenize{examples/getting_started_example:saving-data}}
\sphinxAtStartPar
The framework provides multiple ways to save data, particularly with ready methods to export in \sphinxcode{\sphinxupquote{.csv}} format. It can be
done be calling:

\begin{sphinxVerbatim}[commandchars=\\\{\}]
\PYG{n}{b\PYGZus{}opt}\PYG{o}{.}\PYG{n}{export\PYGZus{}csv}\PYG{p}{(}\PYG{l+s+s1}{\PYGZsq{}}\PYG{l+s+s1}{my\PYGZus{}data.csv}\PYG{l+s+s1}{\PYGZsq{}}\PYG{p}{)}
\end{sphinxVerbatim}

\sphinxAtStartPar
Alternately, data can be custom exported, as \sphinxcode{\sphinxupquote{get}} methods exist to get the population and the respective function values.

\begin{sphinxVerbatim}[commandchars=\\\{\}]
\PYG{n}{total\PYGZus{}population}\PYG{p}{,} \PYG{n}{function\PYGZus{}values} \PYG{o}{=} \PYG{n}{b\PYGZus{}opt}\PYG{o}{.}\PYG{n}{get\PYGZus{}total\PYGZus{}population}\PYG{p}{(}\PYG{p}{)}
\end{sphinxVerbatim}

\sphinxAtStartPar
The next example shows how to custom change the various functions used in the optimization method.


\section{Custom functions for surrogate construction}
\label{\detokenize{examples/user_defined_function_example:custom-functions-for-surrogate-construction}}\label{\detokenize{examples/user_defined_function_example:example-1}}\label{\detokenize{examples/user_defined_function_example::doc}}
\sphinxAtStartPar
This example discusses the modularity provided by the framework to change various optimizer and surrogate related
methods. More specifically, we look at changing the following accessory functions:
\begin{itemize}
\item {} 
\sphinxAtStartPar
{\hyperref[\detokenize{examples/user_defined_function_example:ex1-init}]{\sphinxcrossref{\DUrole{std,std-ref}{Initialization strategy}}}}

\item {} 
\sphinxAtStartPar
{\hyperref[\detokenize{examples/user_defined_function_example:ex1-kernel}]{\sphinxcrossref{\DUrole{std,std-ref}{Kernel functions (Class)}}}}

\item {} 
\sphinxAtStartPar
{\hyperref[\detokenize{examples/user_defined_function_example:ex1-acq}]{\sphinxcrossref{\DUrole{std,std-ref}{Acquisition functions}}}}

\item {} 
\sphinxAtStartPar
{\hyperref[\detokenize{examples/user_defined_function_example:ex1-acq-opt}]{\sphinxcrossref{\DUrole{std,std-ref}{Acquisition optimizer}}}}

\end{itemize}

\sphinxAtStartPar
Through this tutorial, we shall not solve any new problem. However, explanations will be provided to how can one add
custom definitions to these standard methods. We shall use an external implementation of the standard methods for the
user to easily compare with {\hyperref[\detokenize{examples/getting_started_example:example-0}]{\sphinxcrossref{\DUrole{std,std-ref}{Getting Started}}}}. While the example shows the usage of all of these at once, the user is
encouraged to understand the effect of changing each of these accessory functions separately:


\subsection{Initialization strategy}
\label{\detokenize{examples/user_defined_function_example:initialization-strategy}}\label{\detokenize{examples/user_defined_function_example:ex1-init}}
\sphinxAtStartPar
Several situations arise when the user has to specify his own initialization strategies. Some of these include:
\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstylestrong{Constraints}, such as equality, inequality and PDE based constraints. In such situations, all locations determined
by the upper and lower bounds may not be feasible. PARyOpt requires all the initial evaluation points to be successful
evaluations, hence a general latin hypercube sampling may not work in all cases.

\item {} 
\sphinxAtStartPar
\sphinxstylestrong{Biased sampling} of search domain. There could be situations where the user knows strategic locations and thus can
help PARyOpt to sample in those locations right from the beginning

\item {} 
\sphinxAtStartPar
\sphinxstylestrong{User defined sampling strategy}. The user may want a completely different sampling strategy which has a better
coverage of the domain for that specific optimization problem.

\end{itemize}

\sphinxAtStartPar
In order to specify custom initialization strategy, the user should define a function with the following signature:

\begin{sphinxVerbatim}[commandchars=\\\{\}]
\PYG{k}{def} \PYG{n+nf}{my\PYGZus{}init\PYGZus{}strategy}\PYG{p}{(}\PYG{n}{n\PYGZus{}dim}\PYG{p}{:} \PYG{n+nb}{int}\PYG{p}{,} \PYG{n}{n\PYGZus{}init}\PYG{p}{:} \PYG{n+nb}{int}\PYG{p}{,} \PYG{n}{l\PYGZus{}bound}\PYG{p}{:} \PYG{n}{np}\PYG{o}{.}\PYG{n}{array}\PYG{p}{,} \PYG{n}{u\PYGZus{}bound}\PYG{p}{:} \PYG{n}{np}\PYG{o}{.}\PYG{n}{array}\PYG{p}{)} \PYG{o}{\PYGZhy{}}\PYG{o}{\PYGZgt{}} \PYG{n}{List}\PYG{p}{[}\PYG{n}{np}\PYG{o}{.}\PYG{n}{array}\PYG{p}{]}
\end{sphinxVerbatim}

\sphinxAtStartPar
and then pass it during initialization to the parameter \sphinxcode{\sphinxupquote{init\_strategy=my\_init\_strategy}}.


\subsection{Kernel functions (Class)}
\label{\detokenize{examples/user_defined_function_example:kernel-functions-class}}\label{\detokenize{examples/user_defined_function_example:ex1-kernel}}
\sphinxAtStartPar
Kernel function embeds most of the information about the \sphinxstylestrong{continuity} of the underlying function. Thus, it is one of the
critical parameters to be selected by the user. For example, Matern class of kernel functions have only finite
differentiability while the squared exponential kernel function is infinitely differentiable. Some other situations
where the user may want to change the kernel function are :
\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstylestrong{Underlying function continuity and differentiability}, as discussed above.

\item {} 
\sphinxAtStartPar
\sphinxstylestrong{Prior information about periodicity} of the underlying function. In such situations, periodicity can be embedded
into the kernel function. This can drastically improve the number of function evaluations. One such example of a
periodic kernel function is:

\end{itemize}
\begin{equation*}
\begin{split}k_{periodic}(x_1, x_2) = \theta_0 exp(-2 sin(\frac{\pi}{p} \frac{||x_1 - x_2||}{l})^2)\end{split}
\end{equation*}
\sphinxAtStartPar
where \(p\) is the periodicity interval and \(l\) is the length scale of the kernel.
\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstylestrong{Anisotropic kernels} may be used when the function behaviour is different across different optimization parameters.
For example, if the function varies logarithmically in one parameter and linearly in another parameter

\end{itemize}

\sphinxAtStartPar
To specify custom kernel functions, the user has to derive the base kernel class. An example to specify the standard
Matern 3/2 function by the user is as follows:

\begin{sphinxVerbatim}[commandchars=\\\{\}]
\PYG{k}{class} \PYG{n+nc}{MyKernelFunction}\PYG{p}{(}\PYG{n}{KernelFunction}\PYG{p}{)}\PYG{p}{:}
 \PYG{l+s+sd}{\PYGZdq{}\PYGZdq{}\PYGZdq{}}
\PYG{l+s+sd}{ user customized kernel function}
\PYG{l+s+sd}{ \PYGZdq{}\PYGZdq{}\PYGZdq{}}

 \PYG{n}{theta} \PYG{o}{=} \PYG{l+m+mf}{1.0}

 \PYG{k}{def} \PYG{n+nf}{eval}\PYG{p}{(}\PYG{n+nb+bp}{self}\PYG{p}{,} \PYG{n}{x1}\PYG{p}{:} \PYG{n}{np}\PYG{o}{.}\PYG{n}{array}\PYG{p}{,} \PYG{n}{x2}\PYG{p}{:} \PYG{n}{np}\PYG{o}{.}\PYG{n}{array}\PYG{p}{)} \PYG{o}{\PYGZhy{}}\PYG{o}{\PYGZgt{}} \PYG{n+nb}{float}\PYG{p}{:}
     \PYG{l+s+sd}{\PYGZdq{}\PYGZdq{}\PYGZdq{}}
\PYG{l+s+sd}{     actual kernel function}
\PYG{l+s+sd}{     :param x1:}
\PYG{l+s+sd}{     :param x2:}
\PYG{l+s+sd}{     :return:}
\PYG{l+s+sd}{     \PYGZdq{}\PYGZdq{}\PYGZdq{}}
     \PYG{n}{x1} \PYG{o}{=} \PYG{n}{np}\PYG{o}{.}\PYG{n}{divide}\PYG{p}{(}\PYG{n}{x1}\PYG{p}{,} \PYG{n+nb+bp}{self}\PYG{o}{.}\PYG{n}{theta}\PYG{p}{)}
     \PYG{n}{x2} \PYG{o}{=} \PYG{n}{np}\PYG{o}{.}\PYG{n}{divide}\PYG{p}{(}\PYG{n}{x2}\PYG{p}{,} \PYG{n+nb+bp}{self}\PYG{o}{.}\PYG{n}{theta}\PYG{p}{)}
     \PYG{n}{dist} \PYG{o}{=} \PYG{n}{utils}\PYG{o}{.}\PYG{n}{distance}\PYG{p}{(}\PYG{n}{x1}\PYG{p}{,} \PYG{n}{x2}\PYG{p}{)}
     \PYG{n}{rval} \PYG{o}{=} \PYG{n}{np}\PYG{o}{.}\PYG{n}{sqrt}\PYG{p}{(}\PYG{l+m+mf}{3.0}\PYG{p}{)} \PYG{o}{*} \PYG{n}{dist}
     \PYG{k}{return} \PYG{n+nb+bp}{self}\PYG{o}{.}\PYG{n}{theta0} \PYG{o}{*} \PYG{p}{(}\PYG{l+m+mi}{1}\PYG{o}{+}\PYG{n}{rval}\PYG{p}{)} \PYG{o}{*} \PYG{n}{np}\PYG{o}{.}\PYG{n}{exp}\PYG{p}{(}\PYG{o}{\PYGZhy{}}\PYG{n}{rval}\PYG{p}{)}

 \PYG{k}{def} \PYG{n+nf}{derivative}\PYG{p}{(}\PYG{n+nb+bp}{self}\PYG{p}{,} \PYG{n}{x1}\PYG{p}{:} \PYG{n}{np}\PYG{o}{.}\PYG{n}{array}\PYG{p}{,} \PYG{n}{x2}\PYG{p}{:} \PYG{n}{np}\PYG{o}{.}\PYG{n}{array}\PYG{p}{)} \PYG{o}{\PYGZhy{}}\PYG{o}{\PYGZgt{}} \PYG{n}{np}\PYG{o}{.}\PYG{n}{array}\PYG{p}{:}
     \PYG{l+s+sd}{\PYGZdq{}\PYGZdq{}\PYGZdq{}}
\PYG{l+s+sd}{     derivative of kernel function}
\PYG{l+s+sd}{     currently not useful, so we will not implement anything here}
\PYG{l+s+sd}{     :param x1:}
\PYG{l+s+sd}{     :param x2:}
\PYG{l+s+sd}{     :return:}
\PYG{l+s+sd}{     \PYGZdq{}\PYGZdq{}\PYGZdq{}}
     \PYG{k}{pass}
\end{sphinxVerbatim}

\sphinxAtStartPar
This derived class requires defining the \sphinxcode{\sphinxupquote{eval()}} and \sphinxcode{\sphinxupquote{derivative()}} methods of the class. The parameter
\sphinxcode{\sphinxupquote{theta}} should contain all the hyper\sphinxhyphen{}parameters, for ex. length scale, related to the supplied kernel. This will
be used by PARyOpt during hyper\sphinxhyphen{}parameter optimization. This kernel class can be passed to the constructor through
the parameter \sphinxcode{\sphinxupquote{kern\_function=MyKernelFunction()}}.


\subsection{Acquisition functions}
\label{\detokenize{examples/user_defined_function_example:acquisition-functions}}\label{\detokenize{examples/user_defined_function_example:ex1-acq}}
\sphinxAtStartPar
Acquisition function define informative regions of the surrogate. Since the software is designed for minimization, areas
of high information should have small acquisition values. In normal circumstances, the user should not need to change
this. However, if the user wants implementing \sphinxstylestrong{safety constraints} and \sphinxstylestrong{biased informativeness}, it can be be done
by passing a user defined function, with the following signature, to the constructor as
\sphinxcode{\sphinxupquote{acq\_func=my\_acquisition\_function}}.

\begin{sphinxVerbatim}[commandchars=\\\{\}]
\PYG{k}{def} \PYG{n+nf}{my\PYGZus{}acquisition\PYGZus{}function}\PYG{p}{(}\PYG{n}{mean}\PYG{p}{:} \PYG{n+nb}{float}\PYG{p}{,} \PYG{n}{variance}\PYG{p}{:} \PYG{n+nb}{float}\PYG{p}{,} \PYG{n}{curr\PYGZus{}best}\PYG{p}{:} \PYG{n+nb}{float} \PYG{o}{=} \PYG{l+m+mf}{0.}\PYG{p}{,} \PYG{n}{kappa}\PYG{p}{:} \PYG{n+nb}{float} \PYG{o}{=} \PYG{l+m+mf}{1.}\PYG{p}{)} \PYG{o}{\PYGZhy{}}\PYG{o}{\PYGZgt{}} \PYG{n+nb}{float}\PYG{p}{:}
\end{sphinxVerbatim}


\subsection{Acquisition optimizer}
\label{\detokenize{examples/user_defined_function_example:acquisition-optimizer}}\label{\detokenize{examples/user_defined_function_example:ex1-acq-opt}}
\sphinxAtStartPar
Bayesian optimization proceeds by evaluating locations with maximum information content. Hence, it requires finding the
optimum (minimum in PARyOpt) of the acquisition function (cheap to evaluate). While the core software comes with a
standard \sphinxcode{\sphinxupquote{Powell}} algorithm, the user may quite often want to change this for a better global optimum. Some of the
reasons include:
\begin{itemize}
\item {} 
\sphinxAtStartPar
\sphinxstylestrong{Dimensionality of optimization} could impose restrictions on the type of optimizer used

\item {} 
\sphinxAtStartPar
\sphinxstylestrong{Heuristic optimization} can be an alternative for multi\sphinxhyphen{}modal functions. Since evaluating the acquisition is fast,
these optimizers will also be efficient.

\item {} 
\sphinxAtStartPar
Robust \sphinxstylestrong{in\sphinxhyphen{}house optimizers} may be available with research groups tailored for specific problems.

\end{itemize}

\sphinxAtStartPar
The process of adding user\sphinxhyphen{}defined acquisition optimizer is very similar to defining custom acquisition function. The
function signature is :

\begin{sphinxVerbatim}[commandchars=\\\{\}]
\PYG{k}{def} \PYG{n+nf}{my\PYGZus{}acq\PYGZus{}optimizer}\PYG{p}{(}\PYG{n}{func}\PYG{p}{:} \PYG{n}{Callable}\PYG{p}{[}\PYG{p}{[}\PYG{n}{np}\PYG{o}{.}\PYG{n}{array}\PYG{p}{]}\PYG{p}{,} \PYG{n+nb}{float}\PYG{p}{]}\PYG{p}{,} \PYG{n}{x0}\PYG{p}{:} \PYG{n}{np}\PYG{o}{.}\PYG{n}{array}\PYG{p}{,} \PYG{n}{l\PYGZus{}bound}\PYG{p}{:} \PYG{n}{np}\PYG{o}{.}\PYG{n}{array}\PYG{p}{,} \PYG{n}{u\PYGZus{}bound}\PYG{p}{:} \PYG{n}{np}\PYG{o}{.}\PYG{n}{array}\PYG{p}{)} \PYG{o}{\PYGZhy{}}\PYG{o}{\PYGZgt{}} \PYG{n}{np}\PYG{o}{.}\PYG{n}{array}\PYG{p}{:}
\end{sphinxVerbatim}

\sphinxAtStartPar
which is passed to the constructor as \sphinxcode{\sphinxupquote{acq\_func\_optimizer=my\_acq\_optimizer}}.


\section{Restart from previous state}
\label{\detokenize{examples/restart_example:restart-from-previous-state}}\label{\detokenize{examples/restart_example:example-2}}\label{\detokenize{examples/restart_example::doc}}
\sphinxAtStartPar
This example shows how the user can deal with optimization failures, either due to hardware failure or due to wrong
selection of accessory functions. This will also be useful in cases of changing optimization platform but resuming the
same optimization task.

\sphinxAtStartPar
In order to create an optimization state, we shall first run a standard optimization problem for a certain number of
iterations. This can be done either by running example 0({\hyperref[\detokenize{examples/getting_started_example:example-0}]{\sphinxcrossref{\DUrole{std,std-ref}{Getting Started}}}}), or by re\sphinxhyphen{}doing the whole procedure. For the benefit
of the user, we shall take the latter way.


\subsection{Restart}
\label{\detokenize{examples/restart_example:restart}}
\sphinxAtStartPar
In example 1, we have seen that our custom initialization is not the best compared to the default latin hypercube
sampling. In fact, such an example provides the best motivation for a restart. Hence, in this example, we provide
a custom initialization method, the same as in example 1 ({\hyperref[\detokenize{examples/user_defined_function_example:example-1}]{\sphinxcrossref{\DUrole{std,std-ref}{Custom functions for surrogate construction}}}}). Once the optimization is done for 10
iterations, we shall create another instance of \sphinxcode{\sphinxupquote{BayesOpt}} that starts from this existing optimization state.

\begin{sphinxVerbatim}[commandchars=\\\{\}]
\PYG{n}{restarted\PYGZus{}bo} \PYG{o}{=} \PYG{n}{BayesOpt}\PYG{p}{(}\PYG{n}{cost\PYGZus{}function}\PYG{o}{=}\PYG{n}{evaluator}\PYG{p}{,}
                        \PYG{n}{l\PYGZus{}bound}\PYG{o}{=}\PYG{n}{l\PYGZus{}bound}\PYG{p}{,} \PYG{n}{u\PYGZus{}bound}\PYG{o}{=}\PYG{n}{u\PYGZus{}bound}\PYG{p}{,} \PYG{n}{n\PYGZus{}dim}\PYG{o}{=}\PYG{n}{n\PYGZus{}dim}\PYG{p}{,}
                        \PYG{n}{n\PYGZus{}init}\PYG{o}{=}\PYG{l+m+mi}{2}\PYG{p}{,}
                        \PYG{n}{kern\PYGZus{}function}\PYG{o}{=}\PYG{l+s+s1}{\PYGZsq{}}\PYG{l+s+s1}{sqr\PYGZus{}exp}\PYG{l+s+s1}{\PYGZsq{}}\PYG{p}{,}
                        \PYG{n}{acq\PYGZus{}func}\PYG{o}{=}\PYG{l+s+s1}{\PYGZsq{}}\PYG{l+s+s1}{LCB}\PYG{l+s+s1}{\PYGZsq{}}\PYG{p}{,}
                        \PYG{n}{kappa\PYGZus{}strategy}\PYG{o}{=}\PYG{n}{my\PYGZus{}kappa}\PYG{p}{,}
                        \PYG{n}{if\PYGZus{}restart}\PYG{o}{=}\PYG{k+kc}{True}\PYG{p}{,} \PYG{n}{restart\PYGZus{}filename}\PYG{o}{=}\PYG{l+s+s1}{\PYGZsq{}}\PYG{l+s+s1}{opt\PYGZus{}state.dat}\PYG{l+s+s1}{\PYGZsq{}}\PYG{p}{)}
\end{sphinxVerbatim}

\sphinxAtStartPar
Note that the restarted optimization need not have the same accessory functions, like kernel, acquisition and kappa
strategy. By enabling \sphinxcode{\sphinxupquote{if\_restart}} and providing the \sphinxcode{\sphinxupquote{restart\_filename}}, the framework re\sphinxhyphen{}creates the optimization
state from which the user can continue the optimization, for example,

\begin{sphinxVerbatim}[commandchars=\\\{\}]
\PYG{n}{restarted\PYGZus{}bo}\PYG{o}{.}\PYG{n}{update\PYGZus{}iter}\PYG{p}{(}\PYG{l+m+mi}{5}\PYG{p}{)}
\end{sphinxVerbatim}

\sphinxAtStartPar
will update 5 iterations at once. This API helps to reduce redundant loops in the user code.


\subsection{Intra\sphinxhyphen{}iteration restart}
\label{\detokenize{examples/restart_example:intra-iteration-restart}}
\sphinxAtStartPar
It has to be noted here that the evaluator also has an inbuilt check\sphinxhyphen{}pointing per iteration, so that hard interrupts
such as the \sphinxcode{\sphinxupquote{KeyboardInterrupt}} can also be handled for restart. The user need not do any extra changes to enable
this \sphinxstylestrong{intra\sphinxhyphen{}iteration} restart functionality.


\section{Local asynchronous evaluator}
\label{\detokenize{examples/async_local_evaluator_example:local-asynchronous-evaluator}}\label{\detokenize{examples/async_local_evaluator_example:example-3}}\label{\detokenize{examples/async_local_evaluator_example::doc}}
\sphinxAtStartPar
In this example, we shall see how to use the local async evaluator. This is particularly useful when the cost function
takes a long time to evaluate, and sometimes with uncertain evaluation times. In such situations, a better way to
parallelize evaluations is to run as individual jobs on the evaluation platform (local machine) and keep track of
the completion of the jobs through the process id in the process table.
This approach naturally supports asynchronous bayesian update, i.e., update the iteration without completing all the
jobs per iteration. The un\sphinxhyphen{}assimilated jobs will be used for the update in the subsequent iterations.

\sphinxAtStartPar
To achieve this, the framework creates individual job directories for each cost function evaluation. It also provides
an interface for the user to set up these directories for a \sphinxstyleemphasis{self\sphinxhyphen{}contained} evaluation. The cost function needs
to write out a file with the result(cost value) which will later be parsed by the framework upon execution completion.
For this, the user needs to provide functions that
\begin{enumerate}
\sphinxsetlistlabels{\arabic}{enumi}{enumii}{}{.}%
\item {} 
\sphinxAtStartPar
generate necessary files in the folder for function evaluation

\item {} 
\sphinxAtStartPar
provide the command for executing the cost function, and

\item {} 
\sphinxAtStartPar
parse the result file generated by the cost function

\end{enumerate}

\sphinxAtStartPar
In the rest of the document, we shall see how to set up a sample cost function, a folder generator, run command and
result parser. Once again, we shall use the same parabolic cost function for easy understanding, but in 2 dimensions.


\subsection{Out\sphinxhyphen{}of\sphinxhyphen{}script cost function}
\label{\detokenize{examples/async_local_evaluator_example:out-of-script-cost-function}}
\sphinxAtStartPar
The first step to performing out\sphinxhyphen{}of\sphinxhyphen{}script evaluation is to re\sphinxhyphen{}define the cost function. While a generic cost function
in an optimization framework is like a python function that returns the cost function value, out\sphinxhyphen{}of\sphinxhyphen{}script functions
need to be defined differently. Firstly, they do not have any defined arguments and secondly, such out\sphinxhyphen{}of\sphinxhyphen{}script cost
function also cannot directly pass the function value to the optimization framework. While there are several ways to
overcome these difficulties, this framework requires the following template to be followed:
\begin{itemize}
\item {} 
\sphinxAtStartPar
Input: the cost function can either take inline arguments or read from file

\item {} 
\sphinxAtStartPar
Output: the cost function should write to a standard file

\end{itemize}

\sphinxAtStartPar
For example, the following code evaluates the parabola in 2 dimensions and writes to file \sphinxcode{\sphinxupquote{result.txt}}. Location of
evaluation are passed as inline arguments.

\begin{sphinxVerbatim}[commandchars=\\\{\}]
\PYG{k+kn}{import} \PYG{n+nn}{sys}
\PYG{k+kn}{import} \PYG{n+nn}{numpy} \PYG{k}{as} \PYG{n+nn}{np}
\PYG{k+kn}{import} \PYG{n+nn}{time}
\PYG{k+kn}{import} \PYG{n+nn}{examples}\PYG{n+nn}{.}\PYG{n+nn}{examples\PYGZus{}all\PYGZus{}functions} \PYG{k}{as} \PYG{n+nn}{exf}

\PYG{c+c1}{\PYGZsh{} read the command line arguments into an array}
\PYG{n}{xs} \PYG{o}{=} \PYG{n}{np}\PYG{o}{.}\PYG{n}{asarray}\PYG{p}{(}\PYG{p}{[}\PYG{n+nb}{float}\PYG{p}{(}\PYG{n}{x}\PYG{p}{)} \PYG{k}{for} \PYG{n}{x} \PYG{o+ow}{in} \PYG{n}{sys}\PYG{o}{.}\PYG{n}{argv}\PYG{p}{[}\PYG{l+m+mi}{1}\PYG{p}{:}\PYG{p}{]}\PYG{p}{]}\PYG{p}{)}

\PYG{c+c1}{\PYGZsh{} evaluate the cost, i.e. the parabola}
\PYG{n}{cost} \PYG{o}{=} \PYG{n}{exf}\PYG{o}{.}\PYG{n}{parabolic\PYGZus{}cost\PYGZus{}function}\PYG{p}{(}\PYG{n}{x}\PYG{o}{=}\PYG{n}{xs}\PYG{p}{)}

\PYG{c+c1}{\PYGZsh{} In order to demonstrate uncertain evaluation times, we shall use a random sleep in each cost function.}
\PYG{c+c1}{\PYGZsh{} In this case, each function evaluation can take between 0\PYGZhy{}10 sec}
\PYG{n}{time}\PYG{o}{.}\PYG{n}{sleep}\PYG{p}{(}\PYG{n}{np}\PYG{o}{.}\PYG{n}{random}\PYG{o}{.}\PYG{n}{random}\PYG{p}{(}\PYG{p}{)} \PYG{o}{*} \PYG{l+m+mf}{10.}\PYG{p}{)}

\PYG{c+c1}{\PYGZsh{} Write into a result file. Note that this script is evaluated in its respective folder and so}
\PYG{c+c1}{\PYGZsh{} the result.txt file will be in the generated folder and not the home directory of running example4.py}
\PYG{k}{with} \PYG{n+nb}{open}\PYG{p}{(}\PYG{l+s+s1}{\PYGZsq{}}\PYG{l+s+s1}{result.txt}\PYG{l+s+s1}{\PYGZsq{}}\PYG{p}{,} \PYG{l+s+s1}{\PYGZsq{}}\PYG{l+s+s1}{w}\PYG{l+s+s1}{\PYGZsq{}}\PYG{p}{)} \PYG{k}{as} \PYG{n}{f}\PYG{p}{:}
    \PYG{n}{f}\PYG{o}{.}\PYG{n}{write}\PYG{p}{(}\PYG{n+nb}{str}\PYG{p}{(}\PYG{n}{cost}\PYG{p}{)} \PYG{o}{+} \PYG{l+s+s1}{\PYGZsq{}}\PYG{l+s+se}{\PYGZbs{}n}\PYG{l+s+s1}{\PYGZsq{}}\PYG{p}{)}
\end{sphinxVerbatim}


\subsection{Folder generator}
\label{\detokenize{examples/async_local_evaluator_example:folder-generator}}
\sphinxAtStartPar
Many common simulations require not just the location of evaluation but also several other systems to be in place for
proper working. For example, many finite element simulations require a geometry mesh file that represent the domain of
simulation. The optimizer calls this function, \sphinxcode{\sphinxupquote{job\_generator()}} with two arguments \textendash{} the folder of evaluation
(more to come on this) and the location \sphinxcode{\sphinxupquote{x}} at which the cost function is executed.

\begin{sphinxVerbatim}[commandchars=\\\{\}]
\PYG{k}{def} \PYG{n+nf}{folder\PYGZus{}generator}\PYG{p}{(}\PYG{n}{directory}\PYG{p}{,} \PYG{n}{x}\PYG{p}{)} \PYG{o}{\PYGZhy{}}\PYG{o}{\PYGZgt{}} \PYG{k+kc}{None}\PYG{p}{:}
    \PYG{l+s+sd}{\PYGZdq{}\PYGZdq{}\PYGZdq{}}
\PYG{l+s+sd}{    prepares a given folder for performing the simulations. The cost function (out\PYGZhy{}of\PYGZhy{}script) will be executed}
\PYG{l+s+sd}{    in this directory for location x. Typically this involves writing a config file, generating/copying meshes and}

\PYG{l+s+sd}{    In our example, we are running a simple case and so does not require any files to be filled. We shall pass the}
\PYG{l+s+sd}{    location of cost function as a command line argument}
\PYG{l+s+sd}{    \PYGZdq{}\PYGZdq{}\PYGZdq{}}
    \PYG{k}{with} \PYG{n+nb}{open}\PYG{p}{(}\PYG{n}{os}\PYG{o}{.}\PYG{n}{path}\PYG{o}{.}\PYG{n}{join}\PYG{p}{(}\PYG{n}{directory}\PYG{p}{,} \PYG{l+s+s1}{\PYGZsq{}}\PYG{l+s+s1}{config.txt}\PYG{l+s+s1}{\PYGZsq{}}\PYG{p}{)}\PYG{p}{,} \PYG{l+s+s1}{\PYGZsq{}}\PYG{l+s+s1}{w}\PYG{l+s+s1}{\PYGZsq{}}\PYG{p}{)} \PYG{k}{as} \PYG{n}{f}\PYG{p}{:}
        \PYG{k}{pass}  \PYG{c+c1}{\PYGZsh{} write file}
    \PYG{k}{pass}
\end{sphinxVerbatim}


\subsection{Run command}
\label{\detokenize{examples/async_local_evaluator_example:run-command}}
\sphinxAtStartPar
Just like a folder of files for execution, the user may need to provide command line arguments to the cost function
during execution. To achieve this, the optimizer calls the function \sphinxcode{\sphinxupquote{run\_cmd\_generator()}} with the folder of evaluation and
location \sphinxcode{\sphinxupquote{x}} at which the cost function has to be evaluated. Thus it can allow change of run\sphinxhyphen{}time arguments based on
the evaluation point.

\begin{sphinxVerbatim}[commandchars=\\\{\}]
\PYG{k}{def} \PYG{n+nf}{run\PYGZus{}cmd}\PYG{p}{(}\PYG{n}{directory}\PYG{p}{,} \PYG{n}{x}\PYG{p}{)} \PYG{o}{\PYGZhy{}}\PYG{o}{\PYGZgt{}} \PYG{n}{List}\PYG{p}{[}\PYG{n}{Any}\PYG{p}{]}\PYG{p}{:}
    \PYG{l+s+sd}{\PYGZdq{}\PYGZdq{}\PYGZdq{}}
\PYG{l+s+sd}{    Command to run on local machine to get the value of cost function at x, in directory.}
\PYG{l+s+sd}{    In this example, we shall run the script example3\PYGZus{}evaluator.py with the location as an argument.}
\PYG{l+s+sd}{    \PYGZdq{}\PYGZdq{}\PYGZdq{}}
    \PYG{n}{eval\PYGZus{}path} \PYG{o}{=} \PYG{n}{os}\PYG{o}{.}\PYG{n}{path}\PYG{o}{.}\PYG{n}{join}\PYG{p}{(}\PYG{n}{os}\PYG{o}{.}\PYG{n}{path}\PYG{o}{.}\PYG{n}{dirname}\PYG{p}{(}\PYG{n}{os}\PYG{o}{.}\PYG{n}{path}\PYG{o}{.}\PYG{n}{abspath}\PYG{p}{(}\PYG{n+nv+vm}{\PYGZus{}\PYGZus{}file\PYGZus{}\PYGZus{}}\PYG{p}{)}\PYG{p}{)}\PYG{p}{,} \PYG{l+s+s1}{\PYGZsq{}}\PYG{l+s+s1}{example3\PYGZus{}evaluator.py}\PYG{l+s+s1}{\PYGZsq{}}\PYG{p}{)}
    \PYG{k}{return} \PYG{p}{[}\PYG{n}{sys}\PYG{o}{.}\PYG{n}{executable}\PYG{p}{,} \PYG{n}{eval\PYGZus{}path}\PYG{p}{]} \PYG{o}{+} \PYG{n+nb}{list}\PYG{p}{(}\PYG{n}{x}\PYG{p}{[}\PYG{p}{:}\PYG{p}{]}\PYG{p}{)}
\end{sphinxVerbatim}


\subsection{Result parser}
\label{\detokenize{examples/async_local_evaluator_example:result-parser}}
\sphinxAtStartPar
Once the cost function writes the cost value into a file, the result parser is supposed to read and return that cost
value to the optimizer. Some local post processing operations can go into this function. Care should be taken to return
\sphinxstylestrong{only float values}, otherwise it can lead to Type inconsistencies in the optimization routine. The function signature
is the same as that for \sphinxcode{\sphinxupquote{run\_cmd\_generator()}} and \sphinxcode{\sphinxupquote{job\_generator()}}

\begin{sphinxVerbatim}[commandchars=\\\{\}]
\PYG{k}{def} \PYG{n+nf}{result\PYGZus{}parser}\PYG{p}{(}\PYG{n}{directory}\PYG{p}{,} \PYG{n}{x}\PYG{p}{)} \PYG{o}{\PYGZhy{}}\PYG{o}{\PYGZgt{}} \PYG{n+nb}{float}\PYG{p}{:}
    \PYG{l+s+sd}{\PYGZdq{}\PYGZdq{}\PYGZdq{}}
\PYG{l+s+sd}{    Parses the result from a file and returns the cost function.}
\PYG{l+s+sd}{    The file is written be the actual cost function. One can also do post processing in this function and return the}
\PYG{l+s+sd}{    subsequent value. Based on the construct of our cost function example3\PYGZus{}evaluator.py, the generated result.txt}
\PYG{l+s+sd}{    will be in this \PYGZsq{}directory\PYGZsq{}}
\PYG{l+s+sd}{    \PYGZdq{}\PYGZdq{}\PYGZdq{}}
    \PYG{k}{with} \PYG{n+nb}{open}\PYG{p}{(}\PYG{n}{os}\PYG{o}{.}\PYG{n}{path}\PYG{o}{.}\PYG{n}{join}\PYG{p}{(}\PYG{n}{directory}\PYG{p}{,} \PYG{l+s+s1}{\PYGZsq{}}\PYG{l+s+s1}{result.txt}\PYG{l+s+s1}{\PYGZsq{}}\PYG{p}{)}\PYG{p}{,} \PYG{l+s+s1}{\PYGZsq{}}\PYG{l+s+s1}{r}\PYG{l+s+s1}{\PYGZsq{}}\PYG{p}{)} \PYG{k}{as} \PYG{n}{f}\PYG{p}{:}
        \PYG{k}{return} \PYG{n+nb}{float}\PYG{p}{(}\PYG{n}{f}\PYG{o}{.}\PYG{n}{readline}\PYG{p}{(}\PYG{p}{)}\PYG{p}{)}
\end{sphinxVerbatim}


\subsection{Asynchronous optimization}
\label{\detokenize{examples/async_local_evaluator_example:asynchronous-optimization}}
\sphinxAtStartPar
Once the above functions are created, the only new procedure to use asynchronous evaluations is \sphinxstyleemphasis{setting up the evaluator}.
This requires passing in the the three functions, namely, \sphinxcode{\sphinxupquote{job\_generator()}}, \sphinxcode{\sphinxupquote{run\_cmd\_generator()}} and
\sphinxcode{\sphinxupquote{parse\_result()}}. Along with these, it is optional to pass in the location of function evaluations.
The \sphinxcode{\sphinxupquote{evaluator}} creates separate folders in these directory (relative path) for each cost function evaluation.
Each cost function call is assigned a (randomly named) directory within this specified \sphinxcode{\sphinxupquote{jobs\_dir}} where the
\sphinxcode{\sphinxupquote{run\_cmd}} (from \sphinxcode{\sphinxupquote{run\_cmd\_generator()}}) is called.

\begin{sphinxVerbatim}[commandchars=\\\{\}]
\PYG{n}{evaluator} \PYG{o}{=} \PYG{n}{AsyncLocalEvaluator}\PYG{p}{(}\PYG{n}{job\PYGZus{}generator}\PYG{o}{=}\PYG{n}{folder\PYGZus{}generator}\PYG{p}{,}
                                 \PYG{n}{run\PYGZus{}cmd\PYGZus{}generator}\PYG{o}{=}\PYG{n}{run\PYGZus{}cmd}\PYG{p}{,}
                                 \PYG{n}{parse\PYGZus{}result}\PYG{o}{=}\PYG{n}{result\PYGZus{}parser}\PYG{p}{,}
                                 \PYG{n}{required\PYGZus{}fraction}\PYG{o}{=}\PYG{l+m+mf}{0.5}\PYG{p}{,} \PYG{n}{jobs\PYGZus{}dir}\PYG{o}{=}\PYG{n}{os}\PYG{o}{.}\PYG{n}{path}\PYG{o}{.}\PYG{n}{join}\PYG{p}{(}\PYG{n}{os}\PYG{o}{.}\PYG{n}{getcwd}\PYG{p}{(}\PYG{p}{)}\PYG{p}{,} \PYG{l+s+s1}{\PYGZsq{}}\PYG{l+s+s1}{temp/opt\PYGZus{}jobs}\PYG{l+s+s1}{\PYGZsq{}}\PYG{p}{)}\PYG{p}{)}
\end{sphinxVerbatim}

\sphinxAtStartPar
Since we are using multiple optima per iteration, we can take advantage of it deploy simultaneous exploration and
exploitation in the acquisition function. For example, the following code creates a list of two functions \textendash{} one
exploratory (kappa = 1000) and another exploitatory (kappa = 0.1). This list is then passed to the optimizer, like in
the previous examples.

\begin{sphinxVerbatim}[commandchars=\\\{\}]
\PYG{n}{n\PYGZus{}opt} \PYG{o}{=} \PYG{l+m+mi}{2}
\PYG{n}{my\PYGZus{}kappa\PYGZus{}funcs} \PYG{o}{=} \PYG{p}{[}\PYG{p}{]}
\PYG{n}{my\PYGZus{}kappa\PYGZus{}funcs}\PYG{o}{.}\PYG{n}{append}\PYG{p}{(}\PYG{k}{lambda} \PYG{n}{iter\PYGZus{}num}\PYG{p}{:} \PYG{l+m+mi}{1000}\PYG{p}{)}            \PYG{c+c1}{\PYGZsh{} exploration}
\PYG{n}{my\PYGZus{}kappa\PYGZus{}funcs}\PYG{o}{.}\PYG{n}{append}\PYG{p}{(}\PYG{k}{lambda} \PYG{n}{iter\PYGZus{}num}\PYG{p}{:} \PYG{l+m+mf}{0.1}\PYG{p}{)}             \PYG{c+c1}{\PYGZsh{} exploitation}
\end{sphinxVerbatim}

\sphinxAtStartPar
One can get more crafty in designing these kappa strategies and create a so\sphinxhyphen{}called annealing kappa, one that starts with
a large value and eventually reduces to a small value, at different rates.

\begin{sphinxVerbatim}[commandchars=\\\{\}]
\PYG{k}{for} \PYG{n}{j} \PYG{o+ow}{in} \PYG{n+nb}{range}\PYG{p}{(}\PYG{n}{n\PYGZus{}opt}\PYG{p}{)}\PYG{p}{:}
    \PYG{n}{my\PYGZus{}kappa\PYGZus{}funcs}\PYG{o}{.}\PYG{n}{append}\PYG{p}{(}\PYG{k}{lambda} \PYG{n}{curr\PYGZus{}iter\PYGZus{}num}\PYG{p}{,} \PYG{n}{freq}\PYG{o}{=}\PYG{l+m+mf}{10.}\PYG{o}{*}\PYG{p}{(}\PYG{n}{j}\PYG{o}{*}\PYG{n}{j}\PYG{o}{+}\PYG{l+m+mi}{2}\PYG{p}{)}\PYG{p}{,} \PYG{n}{t\PYGZus{}const}\PYG{o}{=}\PYG{l+m+mf}{0.8}\PYG{o}{/}\PYG{p}{(}\PYG{l+m+mf}{1.} \PYG{o}{+} \PYG{n}{j}\PYG{p}{)}\PYG{p}{:}
                          \PYG{n}{user\PYGZus{}defined\PYGZus{}kappa}\PYG{p}{(}\PYG{n}{curr\PYGZus{}iter\PYGZus{}num}\PYG{p}{,} \PYG{n}{freq}\PYG{o}{=}\PYG{n}{freq}\PYG{p}{,} \PYG{n}{t\PYGZus{}const}\PYG{o}{=}\PYG{n}{t\PYGZus{}const}\PYG{p}{)}\PYG{p}{)}
\end{sphinxVerbatim}

\sphinxAtStartPar
The remaining part of the optimization remains the same, except for the initialization of \sphinxcode{\sphinxupquote{BayesOpt}} object.

\begin{sphinxVerbatim}[commandchars=\\\{\}]
\PYG{n}{b\PYGZus{}opt} \PYG{o}{=} \PYG{n}{BayesOpt}\PYG{p}{(}\PYG{n}{cost\PYGZus{}function}\PYG{o}{=}\PYG{n}{evaluator}\PYG{p}{,}
                 \PYG{n}{n\PYGZus{}dim}\PYG{o}{=}\PYG{n}{n\PYGZus{}dim}\PYG{p}{,} \PYG{n}{n\PYGZus{}opt}\PYG{o}{=}\PYG{n}{n\PYGZus{}opt}\PYG{p}{,} \PYG{n}{n\PYGZus{}init}\PYG{o}{=}\PYG{l+m+mi}{2}\PYG{p}{,}
                 \PYG{n}{u\PYGZus{}bound}\PYG{o}{=}\PYG{n}{u\PYGZus{}bound}\PYG{p}{,} \PYG{n}{l\PYGZus{}bound}\PYG{o}{=}\PYG{n}{l\PYGZus{}bound}\PYG{p}{,}
                 \PYG{n}{kern\PYGZus{}function}\PYG{o}{=}\PYG{l+s+s1}{\PYGZsq{}}\PYG{l+s+s1}{matern\PYGZus{}52}\PYG{l+s+s1}{\PYGZsq{}}\PYG{p}{,}
                 \PYG{n}{acq\PYGZus{}func}\PYG{o}{=}\PYG{l+s+s1}{\PYGZsq{}}\PYG{l+s+s1}{LCB}\PYG{l+s+s1}{\PYGZsq{}}\PYG{p}{,} \PYG{n}{kappa\PYGZus{}strategy}\PYG{o}{=}\PYG{n}{my\PYGZus{}kappa\PYGZus{}funcs}\PYG{p}{,}
                 \PYG{n}{if\PYGZus{}restart}\PYG{o}{=}\PYG{k+kc}{False}\PYG{p}{)}

\PYG{k}{for} \PYG{n}{curr\PYGZus{}iter} \PYG{o+ow}{in} \PYG{n+nb}{range}\PYG{p}{(}\PYG{n}{iter\PYGZus{}max}\PYG{p}{)}\PYG{p}{:}
    \PYG{n}{b\PYGZus{}opt}\PYG{o}{.}\PYG{n}{update\PYGZus{}iter}\PYG{p}{(}\PYG{p}{)}
    \PYG{k}{if} \PYG{o+ow}{not} \PYG{n}{curr\PYGZus{}iter} \PYG{o}{\PYGZpc{}} \PYG{l+m+mi}{2}\PYG{p}{:}
        \PYG{n}{b\PYGZus{}opt}\PYG{o}{.}\PYG{n}{estimate\PYGZus{}best\PYGZus{}kernel\PYGZus{}parameters}\PYG{p}{(}\PYG{n}{theta\PYGZus{}bounds}\PYG{o}{=}\PYG{p}{[}\PYG{p}{[}\PYG{l+m+mf}{0.01}\PYG{p}{,} \PYG{l+m+mi}{10}\PYG{p}{]}\PYG{p}{]}\PYG{p}{)}
    \PYG{n}{exf}\PYG{o}{.}\PYG{n}{visualize\PYGZus{}fit}\PYG{p}{(}\PYG{n}{b\PYGZus{}opt}\PYG{p}{)}
\end{sphinxVerbatim}


\section{Kriging}
\label{\detokenize{examples/kriging_example:kriging}}\label{\detokenize{examples/kriging_example:example-4}}\label{\detokenize{examples/kriging_example::doc}}
\sphinxAtStartPar
Kriging or Gaussian process regression is a method of interpolation for which the interpolated values are modeled by a
Gaussian process governed by prior covariances. In this example, we show how PARyOpt can be used to generate response
surfaces using available data.
As with the previous examples, we shall use the standard parabola as the underlying function to be approximated. There
are several ways to use PARyOpt for Kriging, one of which is shown here. This is possibly the easiest and cleanest way
to perform Kriging using PARyOpt.

\sphinxAtStartPar
\sphinxstylestrong{Data generation}: Since the underlying function is known, we shall generate data by invoking this function
at some random locations within the bounds and storing them in an external file. This is achieved through the
following snippet:

\begin{sphinxVerbatim}[commandchars=\\\{\}]
\PYG{k}{def} \PYG{n+nf}{create\PYGZus{}data\PYGZus{}csv}\PYG{p}{(}\PYG{n}{function}\PYG{p}{:} \PYG{n}{Callable}\PYG{p}{,} \PYG{n}{filename}\PYG{p}{:} \PYG{n+nb}{str}\PYG{p}{,} \PYG{n}{l\PYGZus{}bound}\PYG{p}{:} \PYG{n}{np}\PYG{o}{.}\PYG{n}{array}\PYG{p}{,} \PYG{n}{u\PYGZus{}bound}\PYG{p}{:} \PYG{n}{np}\PYG{o}{.}\PYG{n}{array}\PYG{p}{)} \PYG{o}{\PYGZhy{}}\PYG{o}{\PYGZgt{}} \PYG{k+kc}{None}\PYG{p}{:}
 \PYG{c+c1}{\PYGZsh{} generate some random locations \PYGZhy{}\PYGZhy{} 7}
 \PYG{n}{normalized\PYGZus{}population} \PYG{o}{=} \PYG{n}{np}\PYG{o}{.}\PYG{n}{random}\PYG{o}{.}\PYG{n}{ranf}\PYG{p}{(}\PYG{p}{(}\PYG{l+m+mi}{7}\PYG{p}{,} \PYG{p}{)}\PYG{p}{)}
 \PYG{n}{real\PYGZus{}population} \PYG{o}{=} \PYG{n}{l\PYGZus{}bound} \PYG{o}{+} \PYG{n}{normalized\PYGZus{}population} \PYG{o}{*} \PYG{p}{(}\PYG{n}{u\PYGZus{}bound} \PYG{o}{\PYGZhy{}} \PYG{n}{l\PYGZus{}bound}\PYG{p}{)}
 \PYG{n}{real\PYGZus{}population} \PYG{o}{=} \PYG{p}{[}\PYG{n}{np}\PYG{o}{.}\PYG{n}{asarray}\PYG{p}{(}\PYG{p}{[}\PYG{n}{p}\PYG{p}{]}\PYG{p}{)} \PYG{k}{for} \PYG{n}{p} \PYG{o+ow}{in} \PYG{n}{real\PYGZus{}population}\PYG{p}{]}
 \PYG{c+c1}{\PYGZsh{} evaluate the values}
 \PYG{n}{real\PYGZus{}functions} \PYG{o}{=} \PYG{p}{[}\PYG{n+nb}{float}\PYG{p}{(}\PYG{n}{function}\PYG{p}{(}\PYG{n}{p}\PYG{p}{)}\PYG{p}{)} \PYG{k}{for} \PYG{n}{p} \PYG{o+ow}{in} \PYG{n}{real\PYGZus{}population}\PYG{p}{]}

 \PYG{c+c1}{\PYGZsh{} write into file}
 \PYG{k}{with} \PYG{n+nb}{open}\PYG{p}{(}\PYG{n}{filename}\PYG{p}{,} \PYG{l+s+s1}{\PYGZsq{}}\PYG{l+s+s1}{w}\PYG{l+s+s1}{\PYGZsq{}}\PYG{p}{)} \PYG{k}{as} \PYG{n}{f}\PYG{p}{:}
     \PYG{n}{writer} \PYG{o}{=} \PYG{n}{csv}\PYG{o}{.}\PYG{n}{writer}\PYG{p}{(}\PYG{n}{f}\PYG{p}{,} \PYG{n}{delimiter}\PYG{o}{=}\PYG{l+s+s1}{\PYGZsq{}}\PYG{l+s+s1}{,}\PYG{l+s+s1}{\PYGZsq{}}\PYG{p}{)}
     \PYG{n}{writer}\PYG{o}{.}\PYG{n}{writerow}\PYG{p}{(}\PYG{l+s+s1}{\PYGZsq{}}\PYG{l+s+s1}{x y}\PYG{l+s+s1}{\PYGZsq{}}\PYG{p}{)}
     \PYG{k}{for} \PYG{n}{x}\PYG{p}{,} \PYG{n}{y} \PYG{o+ow}{in} \PYG{n+nb}{zip}\PYG{p}{(}\PYG{n}{real\PYGZus{}population}\PYG{p}{,} \PYG{n}{real\PYGZus{}functions}\PYG{p}{)}\PYG{p}{:}
         \PYG{n}{writer}\PYG{o}{.}\PYG{n}{writerow}\PYG{p}{(}\PYG{n+nb}{list}\PYG{p}{(}\PYG{n}{x}\PYG{p}{)} \PYG{o}{+} \PYG{p}{[}\PYG{n}{y}\PYG{p}{]}\PYG{p}{)}
\end{sphinxVerbatim}

\sphinxAtStartPar
\sphinxstylestrong{Data assimilation} : PARyOpt provides an \sphinxcode{\sphinxupquote{add\_point()}} to add external data manually. The user has to supply
the \sphinxtitleref{x} location and any available \sphinxtitleref{y} values to this method to add data to the \sphinxcode{\sphinxupquote{BayesOpt}} instance.
An example usage of \sphinxcode{\sphinxupquote{add\_point()}} using the above generated data can be:

\begin{sphinxVerbatim}[commandchars=\\\{\}]
\PYG{k}{def} \PYG{n+nf}{load\PYGZus{}from\PYGZus{}csv}\PYG{p}{(}\PYG{n}{b\PYGZus{}opt}\PYG{p}{:} \PYG{n}{BayesOpt}\PYG{p}{,} \PYG{n}{filename}\PYG{p}{:} \PYG{n+nb}{str}\PYG{p}{)} \PYG{o}{\PYGZhy{}}\PYG{o}{\PYGZgt{}} \PYG{n}{BayesOpt}\PYG{p}{:}
 \PYG{l+s+sd}{\PYGZdq{}\PYGZdq{}\PYGZdq{}}
\PYG{l+s+sd}{ load data from csv file and add to PARyOpt}
\PYG{l+s+sd}{ \PYGZdq{}\PYGZdq{}\PYGZdq{}}
 \PYG{k}{with} \PYG{n+nb}{open}\PYG{p}{(}\PYG{n}{filename}\PYG{p}{,} \PYG{l+s+s1}{\PYGZsq{}}\PYG{l+s+s1}{r}\PYG{l+s+s1}{\PYGZsq{}}\PYG{p}{)} \PYG{k}{as} \PYG{n}{csvfile}\PYG{p}{:}
     \PYG{n}{csv\PYGZus{}file\PYGZus{}lines} \PYG{o}{=} \PYG{n}{csv}\PYG{o}{.}\PYG{n}{reader}\PYG{p}{(}\PYG{n}{csvfile}\PYG{p}{,} \PYG{n}{delimiter}\PYG{o}{=}\PYG{l+s+s1}{\PYGZsq{}}\PYG{l+s+s1}{,}\PYG{l+s+s1}{\PYGZsq{}}\PYG{p}{)}
     \PYG{k}{for} \PYG{n}{row\PYGZus{}num}\PYG{p}{,} \PYG{n}{row} \PYG{o+ow}{in} \PYG{n+nb}{enumerate}\PYG{p}{(}\PYG{n}{csv\PYGZus{}file\PYGZus{}lines}\PYG{p}{)}\PYG{p}{:}
         \PYG{k}{if} \PYG{n}{row\PYGZus{}num} \PYG{o}{==} \PYG{l+m+mi}{0}\PYG{p}{:}
             \PYG{c+c1}{\PYGZsh{} skipping the header}
             \PYG{k}{pass}
         \PYG{k}{else}\PYG{p}{:}
             \PYG{n}{b\PYGZus{}opt}\PYG{o}{.}\PYG{n}{add\PYGZus{}point}\PYG{p}{(}\PYG{n}{x}\PYG{o}{=}\PYG{n}{np}\PYG{o}{.}\PYG{n}{asarray}\PYG{p}{(}\PYG{p}{[}\PYG{n+nb}{float}\PYG{p}{(}\PYG{n}{row}\PYG{p}{[}\PYG{l+m+mi}{0}\PYG{p}{]}\PYG{p}{)}\PYG{p}{]}\PYG{p}{)}\PYG{p}{,} \PYG{n}{y}\PYG{o}{=}\PYG{n+nb}{float}\PYG{p}{(}\PYG{n}{row}\PYG{p}{[}\PYG{o}{\PYGZhy{}}\PYG{l+m+mi}{1}\PYG{p}{]}\PYG{p}{)}\PYG{p}{,}
                             \PYG{n}{if\PYGZus{}check\PYGZus{}nearness}\PYG{o}{=}\PYG{k+kc}{True}\PYG{p}{)}
 \PYG{n}{b\PYGZus{}opt}\PYG{o}{.}\PYG{n}{update\PYGZus{}surrogate}\PYG{p}{(}\PYG{p}{)}

 \PYG{k}{return} \PYG{n}{b\PYGZus{}opt}
\end{sphinxVerbatim}

\sphinxAtStartPar
Note that the user has to manually invoke \sphinxcode{\sphinxupquote{update\_surrogate()}}. This is currently for efficiency purposes and hope
to be replaced in the upcoming versions.

\sphinxAtStartPar
Finally, since the user wants to add data manually and does not want the standard initialization required for bayesian
optimization, we provide a switch \sphinxcode{\sphinxupquote{do\_init}} to turn off the initialization. Since there is no cost function to
be optimized, the evaluator should be passed in an empty function for evaluation.

\begin{sphinxVerbatim}[commandchars=\\\{\}]
\PYG{c+c1}{\PYGZsh{} dummy evaluators:}
\PYG{n}{evaluator} \PYG{o}{=} \PYG{n}{FunctionEvaluator}\PYG{p}{(}\PYG{k}{lambda} \PYG{n}{x}\PYG{p}{:} \PYG{l+m+mf}{0.0}\PYG{p}{)}

\PYG{n}{krig} \PYG{o}{=} \PYG{n}{BayesOpt}\PYG{p}{(}\PYG{n}{cost\PYGZus{}function}\PYG{o}{=}\PYG{n}{evaluator}\PYG{p}{,}
                \PYG{n}{l\PYGZus{}bound}\PYG{o}{=}\PYG{n}{l\PYGZus{}bound}\PYG{p}{,} \PYG{n}{u\PYGZus{}bound}\PYG{o}{=}\PYG{n}{u\PYGZus{}bound}\PYG{p}{,} \PYG{n}{n\PYGZus{}dim}\PYG{o}{=}\PYG{l+m+mi}{1}\PYG{p}{,}
                \PYG{n}{n\PYGZus{}init}\PYG{o}{=}\PYG{l+m+mi}{0}\PYG{p}{,} \PYG{n}{do\PYGZus{}init}\PYG{o}{=}\PYG{k+kc}{False}\PYG{p}{,}        \PYG{c+c1}{\PYGZsh{} ensures that initialization is not done.}
                \PYG{n}{kern\PYGZus{}function}\PYG{o}{=}\PYG{l+s+s1}{\PYGZsq{}}\PYG{l+s+s1}{sqr\PYGZus{}exp}\PYG{l+s+s1}{\PYGZsq{}}\PYG{p}{,}
                \PYG{n}{acq\PYGZus{}func}\PYG{o}{=}\PYG{l+s+s1}{\PYGZsq{}}\PYG{l+s+s1}{LCB}\PYG{l+s+s1}{\PYGZsq{}}\PYG{p}{,}
                \PYG{n}{kappa\PYGZus{}strategy}\PYG{o}{=}\PYG{k}{lambda} \PYG{n}{curr\PYGZus{}iter}\PYG{p}{:} \PYG{l+m+mi}{1000}\PYG{p}{,}
                \PYG{n}{if\PYGZus{}restart}\PYG{o}{=}\PYG{k+kc}{False}\PYG{p}{)}
\PYG{n}{krig} \PYG{o}{=} \PYG{n}{load\PYGZus{}from\PYGZus{}csv}\PYG{p}{(}\PYG{n}{krig}\PYG{p}{,} \PYG{n}{data\PYGZus{}filename}\PYG{p}{)}
\PYG{n}{krig}\PYG{o}{.}\PYG{n}{estimate\PYGZus{}best\PYGZus{}kernel\PYGZus{}parameters}\PYG{p}{(}\PYG{n}{theta\PYGZus{}bounds}\PYG{o}{=}\PYG{p}{[}\PYG{p}{[}\PYG{l+m+mf}{0.001}\PYG{p}{,} \PYG{l+m+mf}{10.0}\PYG{p}{]}\PYG{p}{]}\PYG{p}{)}
\end{sphinxVerbatim}

\sphinxAtStartPar
Note that since we are not providing any actual cost function here, \sphinxcode{\sphinxupquote{update\_iter()}} does nothing useful. In case
the user is looking for an instantaneous Kriging model, i.e., creating a Kriging surface and updating it, the actual
cost function should be provided. Just like the previous examples, one may use \sphinxcode{\sphinxupquote{evaluator}} from {\hyperref[\detokenize{examples/async_local_evaluator_example:example-3}]{\sphinxcrossref{\DUrole{std,std-ref}{Local asynchronous evaluator}}}}
and do Kriging similar to optimization.

\sphinxAtStartPar
Now that the surrogate is created and hyper\sphinxhyphen{}parameters optimized, one can start querying it using
\sphinxcode{\sphinxupquote{evaluate\_surrogate\_at()}}

\begin{sphinxVerbatim}[commandchars=\\\{\}]
\PYG{n}{location} \PYG{o}{=} \PYG{n}{np}\PYG{o}{.}\PYG{n}{array}\PYG{p}{(}\PYG{p}{[}\PYG{l+m+mf}{1.0}\PYG{p}{]}\PYG{p}{)}
\PYG{n}{mean}\PYG{p}{,} \PYG{n}{variance} \PYG{o}{=} \PYG{n}{krig}\PYG{o}{.}\PYG{n}{evaluate\PYGZus{}surrogate\PYGZus{}at}\PYG{p}{(}\PYG{n}{location}\PYG{p}{)}
\end{sphinxVerbatim}

\sphinxAtStartPar
PARyOpt (pronounced \sphinxstylestrong{pur\sphinxhyphen{}yopt}, with a hard \sphinxstyleemphasis{y}) is a modular asynchronous Bayesian optimization package that enables
remote function evaluation.

\sphinxAtStartPar
It is written to solve optimization problems where the cost function is very expensive to evaluate
(on the order of hours), and where cost functions must be evaluated on remote machines (HPC clusters).


\chapter{Indices and tables}
\label{\detokenize{index:indices-and-tables}}\begin{itemize}
\item {} 
\sphinxAtStartPar
\DUrole{xref,std,std-ref}{genindex}

\item {} 
\sphinxAtStartPar
\DUrole{xref,std,std-ref}{modindex}

\item {} 
\sphinxAtStartPar
\DUrole{xref,std,std-ref}{search}

\end{itemize}


\renewcommand{\indexname}{Python Module Index}
\begin{sphinxtheindex}
\let\bigletter\sphinxstyleindexlettergroup
\bigletter{c}
\item\relax\sphinxstyleindexentry{conf}\sphinxstyleindexpageref{conf:\detokenize{module-conf}}
\indexspace
\bigletter{p}
\item\relax\sphinxstyleindexentry{PARyOpt}\sphinxstyleindexpageref{PARyOpt:\detokenize{module-PARyOpt}}
\item\relax\sphinxstyleindexentry{PARyOpt.acquisition\_functions}\sphinxstyleindexpageref{PARyOpt:\detokenize{module-PARyOpt.acquisition_functions}}
\item\relax\sphinxstyleindexentry{PARyOpt.evaluators}\sphinxstyleindexpageref{PARyOpt.evaluators:\detokenize{module-PARyOpt.evaluators}}
\item\relax\sphinxstyleindexentry{PARyOpt.evaluators.async\_local}\sphinxstyleindexpageref{PARyOpt.evaluators:\detokenize{module-PARyOpt.evaluators.async_local}}
\item\relax\sphinxstyleindexentry{PARyOpt.evaluators.async\_parse\_result\_local}\sphinxstyleindexpageref{PARyOpt.evaluators:\detokenize{module-PARyOpt.evaluators.async_parse_result_local}}
\item\relax\sphinxstyleindexentry{PARyOpt.evaluators.async\_sbatch}\sphinxstyleindexpageref{PARyOpt.evaluators:\detokenize{module-PARyOpt.evaluators.async_sbatch}}
\item\relax\sphinxstyleindexentry{PARyOpt.evaluators.connection}\sphinxstyleindexpageref{PARyOpt.evaluators:\detokenize{module-PARyOpt.evaluators.connection}}
\item\relax\sphinxstyleindexentry{PARyOpt.evaluators.paryopt\_async}\sphinxstyleindexpageref{PARyOpt.evaluators:\detokenize{module-PARyOpt.evaluators.paryopt_async}}
\item\relax\sphinxstyleindexentry{PARyOpt.kernel}\sphinxstyleindexpageref{PARyOpt.kernel:\detokenize{module-PARyOpt.kernel}}
\item\relax\sphinxstyleindexentry{PARyOpt.kernel.kernel\_function}\sphinxstyleindexpageref{PARyOpt.kernel:\detokenize{module-PARyOpt.kernel.kernel_function}}
\item\relax\sphinxstyleindexentry{PARyOpt.kernel.matern}\sphinxstyleindexpageref{PARyOpt.kernel:\detokenize{module-PARyOpt.kernel.matern}}
\item\relax\sphinxstyleindexentry{PARyOpt.kernel.squared\_exponential}\sphinxstyleindexpageref{PARyOpt.kernel:\detokenize{module-PARyOpt.kernel.squared_exponential}}
\item\relax\sphinxstyleindexentry{PARyOpt.paryopt}\sphinxstyleindexpageref{PARyOpt:\detokenize{module-PARyOpt.paryopt}}
\item\relax\sphinxstyleindexentry{PARyOpt.utils}\sphinxstyleindexpageref{PARyOpt:\detokenize{module-PARyOpt.utils}}
\end{sphinxtheindex}

\renewcommand{\indexname}{Index}
\printindex
\end{document}