L i IgdZdZddlZddlZddlZddlZddlmZmZm Z m Z m Z m Z ddlZ ddlmZmZmZddlmZddlmZmZmZmZdd lmZdd lmZdd lmZmZm Z m!Z!m"Z"dd l#m$Z$m%Z%dd l&m'Z'm(Z(m)Z)ddl*m+Z,dddddddZ-GddZ.d_dZ/Gdde Z0Gdde1Z2dZ3dZ4dZ5e e jle7jpZ9d`d Z: dad"Z;d#Ze(d&g'd(Z?e(d&g'd)Z@e(d&g'd*ZAd+ZBGd,d-eCZDd.ZE dbd0ZF dcd2ZGe9fd3ZHe"d4d56e9d7dd8d9ZId:ZJGd;de je9ddddddd/d?dfd@ZNd!ddd>e je9dd1d1ddd/d?ddfdAZOd_dBZPdd!d>e je9dddddd/dCf dDZQd!ddd>e je9dd1d1dd/dCdf dEZRddd!d>e9dddddd/d?f dFZSd!ddddd>e9dd1d1d/d?df dGZT dddHZU dedIZVGdJdKZWdfdLZX dgdMZYd!de9ddNfdOZZdd!e9dNdfdPZ[dhdQZ\GdRdSeCZ]dTZ^dUZ_ didVZ` dbdWZa djdXZbdYZcd!dZdeFd1dfd[ZdGd\d]Zedkd^Zfy)l)fmin fmin_powell fmin_bfgsfmin_ncgfmin_cg fminboundbrentgoldenbracketrosen rosen_der rosen_hessrosen_hess_prodbrute approx_fprime line_search check_gradOptimizeResult show_optionsOptimizeWarningzrestructuredtext enN)eyeargminzerosshapeasarraysqrt)cholesky issymmetric LinAlgError)LinearOperator)line_search_wolfe1line_search_wolfe2r#LineSearchWarning)approx_derivative)getfullargspec_no_self) MapWrappercheck_random_state _RichResult_call_callback_maybe_halt_transition_to_rng)ScalarFunction FD_METHODS)array_namespacexp_capabilities xp_promote)array_api_extraz%Optimization terminated successfully.z9Maximum number of function evaluations has been exceeded.z/Maximum number of iterations has been exceeded.z=Desired error not necessarily achieved due to precision loss.zNaN result encountered.z-The result is outside of the provided bounds.)successmaxfevmaxiterpr_lossnan out_of_boundsc(eZdZdZdZdZdZdZy) MemoizeJaczkDecorator that caches the return values of a function returning ``(fun, grad)`` each time it is called.c<||_d|_d|_d|_yN)funjac_valuex)selfr<s ^/mnt/ssd/data/python-lab/Trading/venv/lib/python3.12/site-packages/scipy/optimize/_optimize.py__init__zMemoizeJac.__init__As ctj||jk(r|j |jPtj |j |_|j|g|}|d|_|d|_yy)Nr!r)npallr?r>r=rcopyr<)r@r?argsfgs rA_compute_if_neededzMemoizeJac._compute_if_neededGsqvva466k"dkk&9TXX=MZZ]'')DF!#d#B!uDHQ%DK >NrCc@|j|g||jS)z returns the function value )rJr>r@r?rHs rA__call__zMemoizeJac.__call__Ns!)D){{rCc@|j|g||jSr;)rJr=rLs rA derivativezMemoizeJac.derivativeSs )D)xxrCN)__name__ __module__ __qualname____doc__rBrJrMrOrCrAr9r9=s   rCr9c|dvrStj}t|jdhk(rfd}n|dk(rfd}n|dk(rfd}nfd}d |_|S) zAWrap a user-provided callback so that attributes can be attached.>tncslsqpcobylacobyqaintermediate_resultc|S)N)rZrTrescallbacks rAwrapped_callbackz(_wrap_callback..wrapped_callback`s4 4rC trust-constrcPtj|j|Sr;rErGr?r\s rAr_z(_wrap_callback..wrapped_callbackcsBGGCEENC0 0rCdifferential_evolutioncdtj|j|jSr;)rErGr? convergencer\s rAr_z(_wrap_callback..wrapped_callbackfs BGGCEENCOO< .wrapped_callbackisBGGCEEN+ +rCF)inspect signatureset parametersstop_iteration)r^methodsigr_s` rA_wrap_callbackrnXsq6%II   H %C 3>>455 5 > ! 1 + + = ,',# rCceZdZdZy)ra Represents the optimization result. Attributes ---------- x : ndarray The solution of the optimization. success : bool Whether or not the optimizer exited successfully. status : int Termination status of the optimizer. Its value depends on the underlying solver. Refer to `message` for details. message : str Description of the cause of the termination. fun : float Value of objective function at `x`. jac, hess : ndarray Values of objective function's Jacobian and its Hessian at `x` (if available). The Hessian may be an approximation, see the documentation of the function in question. hess_inv : object Inverse of the objective function's Hessian; may be an approximation. Not available for all solvers. The type of this attribute may be either np.ndarray or scipy.sparse.linalg.LinearOperator. nfev, njev, nhev : int Number of evaluations of the objective functions and of its Jacobian and Hessian. nit : int Number of iterations performed by the optimizer. maxcv : float The maximum constraint violation. Notes ----- Depending on the specific solver being used, `OptimizeResult` may not have all attributes listed here, and they may have additional attributes not listed here. Since this class is essentially a subclass of dict with attribute accessors, one can see which attributes are available using the `OptimizeResult.keys` method. NrPrQrRrSrTrCrArrps (R rCrceZdZdZy)rz*General warning for :mod:`scipy.optimize`.NrprTrCrArrs4rCrc6d}|||s tdyy)NcRt|r t|yy#t$rYywxYw)NTF)rrr)As rA is_pos_defz,_check_positive_definite..is_pos_defs1 q>    s  &&z+'hess_inv0' matrix isn't positive definite.) ValueError)Hkrus rA_check_positive_definiterxs+ ~"~JK KrCc|rMdjtt|j}t j d|t dyy)Nz, zUnknown solver options:  stacklevel)joinmapstrkeyswarningswarnr)unknown_optionsmsgs rA_check_unknown_optionsrsCiiC!5!5!789  06TUV rCc`tj|dk(xrtj|S)zJTest whether `x` is either a finite scalar or a finite array scalar. r!)rEsizeisfinite)r?s rAis_finite_scalarrs# 771:? -r{{1~-rCcV|tjk(r(tjtj|S|tj k(r(tjtj|Stj tj||zdd|z zS)Nraxis?)rEinfamaxabsaminsum)r?ords rAvecnormrsn bff}wwrvvay!! wwrvvay!!vvbffQin1-c ::rCrTc t|r|} n|tvrd}|} nd} |}|d}|!tj tjf}|xst}t |||| ||||| } | S)a Creates a ScalarFunction object for use with scalar minimizers (BFGS/LBFGSB/SLSQP/TNC/CG/etc). Parameters ---------- fun : callable The objective function to be minimized. ``fun(x, *args) -> float`` where ``x`` is an 1-D array with shape (n,) and ``args`` is a tuple of the fixed parameters needed to completely specify the function. x0 : ndarray, shape (n,) Initial guess. Array of real elements of size (n,), where 'n' is the number of independent variables. jac : {callable, '2-point', '3-point', 'cs', None}, optional Method for computing the gradient vector. If it is a callable, it should be a function that returns the gradient vector: ``jac(x, *args) -> array_like, shape (n,)`` If one of `{'2-point', '3-point', 'cs'}` is selected then the gradient is calculated with a relative step for finite differences. If `None`, then two-point finite differences with an absolute step is used. args : tuple, optional Extra arguments passed to the objective function and its derivatives (`fun`, `jac` functions). bounds : sequence, optional Bounds on variables. 'new-style' bounds are required. eps : float or ndarray If ``jac is None`` the absolute step size used for numerical approximation of the jacobian via forward differences. finite_diff_rel_step : None or array_like, optional If ``jac in ['2-point', '3-point', 'cs']`` the relative step size to use for numerical approximation of the jacobian. The absolute step size is computed as ``h = rel_step * sign(x0) * max(1, abs(x0))``, possibly adjusted to fit into the bounds. For ``jac='3-point'`` the sign of `h` is ignored. If None (default) then step is selected automatically. hess : {callable, '2-point', '3-point', 'cs', None} Computes the Hessian matrix. If it is callable, it should return the Hessian matrix: ``hess(x, *args) -> {LinearOperator, spmatrix, array}, (n, n)`` Alternatively, the keywords {'2-point', '3-point', 'cs'} select a finite difference scheme for numerical estimation. Whenever the gradient is estimated via finite-differences, the Hessian cannot be estimated with options {'2-point', '3-point', 'cs'} and needs to be estimated using one of the quasi-Newton strategies. workers : int or map-like callable, optional A map-like callable, such as `multiprocessing.Pool.map` for evaluating any numerical differentiation in parallel. This evaluation is carried out as ``workers(fun, iterable)``, or ``workers(grad, iterable)``, depending on what is being numerically differentiated. Alternatively, if `workers` is an int the task is subdivided into `workers` sections and the function evaluated in parallel (uses `multiprocessing.Pool `). Supply -1 to use all available CPU cores. It is recommended that a map-like be used instead of int, as repeated calls to `approx_derivative` will incur large overhead from setting up new processes. .. versionadded:: 1.16.0 Returns ------- sf : ScalarFunction N2-pointcyr;rT)r?rHs rAhessz&_prepare_scalar_function..hess+srC)epsilonworkers)callabler-rErr~r,) r<x0r=rHboundsrfinite_diff_rel_steprrgradsfs rA_prepare_scalar_functionrsV}   |  ~66'266"nG RtT,fg ' )B IrCcfd}|S)Nc,t|}|Sr;) _check_clip_x)r?rfuncs rAevalz_clip_x_for_func..evalCs !V $AwrCrT)rrrs`` rA_clip_x_for_funcr=s  KrCc||dkjs||dkDjr;tjdtdt j ||d|d}|S|S)Nrr!zJValues in x were outside bounds during a minimize step, clipping to boundsr{)anyrrRuntimeWarningrEclip)r?rs rArrJse F1I VAY 3 3 5 :$ 4 GGAvay&) , HrCct|}t|d|}|jd|dd|dddzz dzzd|ddz dzzd|j }|S) a The Rosenbrock function. The function computed is:: sum(100.0*(x[1:] - x[:-1]**2.0)**2.0 + (1 - x[:-1])**2.0) Parameters ---------- x : array_like 1-D array of points at which the Rosenbrock function is to be computed. Returns ------- f : float The value of the Rosenbrock function. See Also -------- rosen_der, rosen_hess, rosen_hess_prod Examples -------- >>> import numpy as np >>> from scipy.optimize import rosen >>> X = 0.1 * np.arange(10) >>> rosen(X) 76.56 For higher-dimensional input ``rosen`` broadcasts. In the following example, we use this to plot a 2D landscape. Note that ``rosen_hess`` does not broadcast in this manner. >>> import matplotlib.pyplot as plt >>> from mpl_toolkits.mplot3d import Axes3D >>> x = np.linspace(-1, 1, 50) >>> X, Y = np.meshgrid(x, x) >>> ax = plt.subplot(111, projection='3d') >>> ax.plot_surface(X, Y, rosen([X, Y])) >>> plt.show() Tforce_floatingxpgY@r!N@r)rdtype)r.r0rr)r?rrs rAr r UssV  B1Tb1A u!"#2 +c11Q3BZ#4EEQWW  &A HrC)z jax.numpyz"JAX doesn't allow item assignment.) skip_backendsc8t|}t|d|}|dd}|dd}|dd}|j|}d||dzz zd ||dzz z|zz dd|z zz |ddd |d z|d|d dzz zdd|d z zz |d <d|d|ddzz z|d<|S) aP The derivative (i.e. gradient) of the Rosenbrock function. Parameters ---------- x : array_like 1-D array of points at which the derivative is to be computed. Returns ------- rosen_der : (N,) ndarray The gradient of the Rosenbrock function at `x`. See Also -------- rosen, rosen_hess, rosen_hess_prod Examples -------- >>> import numpy as np >>> from scipy.optimize import rosen_der >>> X = 0.1 * np.arange(9) >>> rosen_der(X) array([ -2. , 10.6, 15.6, 13.4, 6.4, -3. , -12.4, -19.4, 62. ]) Trr!rNpr)r.r0 zeros_like)r?rxmxm_m1xm_p1ders rAr r s8  B1Tb1A 1RB crFE abEE -- CUAX &A &+,./1r6l;C"I AaD[AaD1Q47N +a1qt8n >> import numpy as np >>> from scipy.optimize import rosen_hess >>> X = 0.1 * np.arange(4) >>> rosen_hess(X) array([[-38., 0., 0., 0.], [ 0., 134., -40., 0.], [ 0., -40., 130., -80.], [ 0., 0., -80., 200.]]) TrrNrr!)offsetrrrrrr)r) r.r0xpxcreate_diagonalrrratri)r?rHdiagonals rAr r s.>  B1Tb1A  TAcrF]1 <   sQsV|B2 > ?Axx !''x2Hvvh"&&tadAg~ad 'BQ'FGHvvh#'',Hvvh"%))#q2wz0A*AC!AB%K*OPH s""83 33rCct||}t|d|}tj|d|}|j ||j }|j |jd|j }d|ddzzd |dzz dz|dzd |dz|dzz |d<d |d d z|d d zd d|dddzzzd |dd zz |ddzzd |ddz|dd zz |ddd |d z|d zd|dzz|d<|S)a Product of the Hessian matrix of the Rosenbrock function with a vector. Parameters ---------- x : array_like 1-D array of points at which the Hessian matrix is to be computed. p : array_like 1-D array, the vector to be multiplied by the Hessian matrix. Returns ------- rosen_hess_prod : ndarray The Hessian matrix of the Rosenbrock function at `x` multiplied by the vector `p`. See Also -------- rosen, rosen_der, rosen_hess Examples -------- >>> import numpy as np >>> from scipy.optimize import rosen_hess_prod >>> X = 0.1 * np.arange(9) >>> p = 0.5 * np.arange(9) >>> rosen_hess_prod(X, p) array([ -0., 27., -10., -95., -192., -265., -278., -195., -180.]) Trr!)ndimrrrrrrrNrrrr)r.r0r atleast_ndrrrr)r?prHps rArrsM@ A B1Tb1A qqR(A 1AGG $A !''!*AGG ,B AaD!G^cAaDj (1 ,! 4sQqTzAaD7H HBqEq"v #2&ta"gqj((312;6!Ab'ABa"g !"%&BqHAbE\AbE !C"I -BrF IrCc0dgdfSfd}|fS)Nrcdxxdz cc<tj|g|z}tj|s& tj|j }|S|S#t t f$r}t d|d}~wwxYw)Nrr!@The user-provided objective function must return a scalar value.)rErGisscalarritem TypeErrorrv)r? wrapper_argsfxerHfunctionncallss rAfunction_wrapperz/_wrap_scalar_function..function_wrappersq Q bggaj 9L4$7 9{{2 GZZ^((* r z* G "?@EFG Gs#A++B : BB rT)rrHrrs`` @rA_wrap_scalar_functionrs/SFt|  # ##rCc eZdZy)_MaxFuncCallErrorNrPrQrRrTrCrArr rCrc4dgdfSfd}|fS)NrcBdk\r tddxxdz cc<tj|g|z}tj|s& tj|j }|S|S#t tf$r}td|d}~wwxYw)NrzToo many function callsr!r)rrErGrrrrrv)r?rrrrHrmaxfunrs rArzA_wrap_scalar_function_maxfun_validation..function_wrapper+s !9 #$=> >q Q bggaj 9L4$7 9{{2 GZZ^((* r z* G "?@EFG Gs#A>>B BBrT)rrHrrrs``` @rA'_wrap_scalar_function_maxfun_validationr$s/SFt|" # ##rC-C6?c |||||| | d} t| } t|||fd| i| } |r#| d| d| d| d| df}| r || dfz }|S| r | d| dfS| dS) a( Minimize a function using the downhill simplex algorithm. This algorithm only uses function values, not derivatives or second derivatives. Parameters ---------- func : callable func(x,*args) The objective function to be minimized. x0 : ndarray Initial guess. args : tuple, optional Extra arguments passed to func, i.e., ``f(x,*args)``. xtol : float, optional Absolute error in xopt between iterations that is acceptable for convergence. ftol : number, optional Absolute error in func(xopt) between iterations that is acceptable for convergence. maxiter : int, optional Maximum number of iterations to perform. maxfun : number, optional Maximum number of function evaluations to make. full_output : bool, optional Set to True if fopt and warnflag outputs are desired. disp : bool, optional Set to True to print convergence messages. retall : bool, optional Set to True to return list of solutions at each iteration. callback : callable, optional Called after each iteration, as callback(xk), where xk is the current parameter vector. initial_simplex : array_like of shape (N + 1, N), optional Initial simplex. If given, overrides `x0`. ``initial_simplex[j,:]`` should contain the coordinates of the jth vertex of the ``N+1`` vertices in the simplex, where ``N`` is the dimension. Returns ------- xopt : ndarray Parameter that minimizes function. fopt : float Value of function at minimum: ``fopt = func(xopt)``. iter : int Number of iterations performed. funcalls : int Number of function calls made. warnflag : int 1 : Maximum number of function evaluations made. 2 : Maximum number of iterations reached. allvecs : list Solution at each iteration. See also -------- minimize: Interface to minimization algorithms for multivariate functions. See the 'Nelder-Mead' `method` in particular. Notes ----- Uses a Nelder-Mead simplex algorithm to find the minimum of function of one or more variables. This algorithm has a long history of successful use in applications. But it will usually be slower than an algorithm that uses first or second derivative information. In practice, it can have poor performance in high-dimensional problems and is not robust to minimizing complicated functions. Additionally, there currently is no complete theory describing when the algorithm will successfully converge to the minimum, or how fast it will if it does. Both the ftol and xtol criteria must be met for convergence. Examples -------- >>> def f(x): ... return x**2 >>> from scipy import optimize >>> minimum = optimize.fmin(f, 1) Optimization terminated successfully. Current function value: 0.000000 Iterations: 17 Function evaluations: 34 >>> minimum[0] -8.8817841970012523e-16 References ---------- .. [1] Nelder, J.A. and Mead, R. (1965), "A simplex method for function minimization", The Computer Journal, 7, pp. 308-313 .. [2] Wright, M.H. (1996), "Direct Search Methods: Once Scorned, Now Respectable", in Numerical Analysis 1995, Proceedings of the 1995 Dundee Biennial Conference in Numerical Analysis, D.F. Griffiths and G.A. Watson (Eds.), Addison Wesley Longman, Harlow, UK, pp. 191-208. )xatolfatolr4r3disp return_allinitial_simplexr^r?r<nitnfevstatusallvecs)rn_minimize_neldermead)rrrHxtolftolr4r full_outputrretallr^roptsr]retlists rArr?sN .  0Dh'H tR I ID ICc(CJE CKXN  I) )G s8S^+ +s8OrCFc Zt| |}|}tj|j}tj|j tj r |j ntj}tj||}| r2tt|}d}dd|z z}ddd|zz z }dd|z z }nd}d}d}d}d}d}| | j| j}}||kDjr td tj||kDstj||kDrtj d t"d | tj$|}|t|}tj&|dz|f|j }||d <t)|D]=}tj*|d}||d k7rd|z||z||<n|||<|||dz<?ntj,|j/}tj|j tj r |j ntj}tj||}|j0dk7s"|j2d |j2ddzk7r tdt||j2dk7r td|j2d}|r|d g}| | |dz}|dz}nW|*|tj4k(r|dz}n$rYEwxYw#tj@|!}#tjB||#d }tjB|!|#d }!wxYw#t>$rYwxYw) aj Minimization of scalar function of one or more variables using the Nelder-Mead algorithm. Options ------- disp : bool Set to True to print convergence messages. maxiter, maxfev : int Maximum allowed number of iterations and function evaluations. Will default to ``N*200``, where ``N`` is the number of variables, if neither `maxiter` or `maxfev` is set. If both `maxiter` and `maxfev` are set, minimization will stop at the first reached. return_all : bool, optional Set to True to return a list of the best solution at each of the iterations. initial_simplex : array_like of shape (N + 1, N) Initial simplex. If given, overrides `x0`. ``initial_simplex[j,:]`` should contain the coordinates of the jth vertex of the ``N+1`` vertices in the simplex, where ``N`` is the dimension. xatol : float, optional Absolute error in xopt between iterations that is acceptable for convergence. fatol : number, optional Absolute error in func(xopt) between iterations that is acceptable for convergence. adaptive : bool, optional Adapt algorithm parameters to dimensionality of problem. Useful for high-dimensional minimization [1]_. bounds : sequence or `Bounds`, optional Bounds on variables. There are two ways to specify the bounds: 1. Instance of `Bounds` class. 2. Sequence of ``(min, max)`` pairs for each element in `x`. None is used to specify no bound. Note that this just clips all vertices in simplex based on the bounds. References ---------- .. [1] Gao, F. and Han, L. Implementing the Nelder-Mead simplex algorithm with adaptive parameters. 2012. Computational Optimization and Applications. 51:1, pp. 259-277 rr!rg??g?gMb0?NzENelder Mead - one of the lower bounds is greater than an upper bound.0Initial guess is not within the specified boundsrr{rT)rGz5`initial_simplex` should be an array of shape (N+1,N)z5Size of `initial_simplex` is not consistent with `x0`rrrr?r<r3r4r2! Current function value: f Iterations: d Function evaluations: )r<rrrr2messager? final_simplexr).rrE atleast_1dflatten issubdtyperinexactfloat64rfloatlenlbubrrvrrrremptyrangearray atleast_2drGrrrwherelistfullrrargsorttakemaxravelraddreduceappendrr*min_status_messagerprint)6rrrHr^r4r3rrrrradaptiverrrrrdimrhochipsisigmanonzdeltzdelt lower_bound upper_boundNsimkyrmskone2np1fsimfcallsind iterationsxbarxrfxrdoshrinkxefxexcfxcxccfxccjrZr?fvalwarnflagrresults6 rArrsl?+ F F r  " " $B bhh ;BHHE Be $BCGn!C%iQ#YAcE H E #)99fii[ + % * * ,?@ @ 66+" #rvvb;.>'? MML)a 9 WWRk 2 GhhAqz2Aq A$'AtqyH ad*!!CAJ  mmO,113]]399bjjA rzzjjE* 88q=CIIaLCIIaL1,<<TU U r7ciil "TU U IIaL q6(6>c'S  RVV #gGffG  bff WFVVF  KhhsAkMC/5ggc; 45AE?#G 77AE8RVV5 1D:4vNLFD%q1u #A3q6lDG # jjggc3"wwtS!$ **T C 774a D ''#sA CJ !9v *w"6? rxxs12wQ'7 89:eCFF266$q'DH"456%?66==Sb1-1Dc'T!C#b'M1B!WWRk:r(CHT!W}#)mt+cCi#b'.AA%[+>B2h9 CG"DH CG"DHb> CG"DHT"X~#)mt3cCi#b'6II!-!#[+!FB"2h#:&(CG'*DH'(H !3w$.s2w>!-"$''#{K"HC#Cy$r(?&)CG'+DH'(H!(3A%(Ves1vA.G%GCF%1)+$'FK*FA&*3q6lDG 3 !OJjjggc3"wwtS!$  NN3q6 ",s1v47C $X/B C Q !9v *w"6T AA 66$ ? )*Q8 9 3F1Ia=A B *6!9#+h!m$'1S$KIF#y Mg    jjggc3"wwtS!$P!   sE*!a a$A;b!F5b aaaaAb b*)b*cjtj|t}||g|}t||d|||S)a Finite difference approximation of the derivatives of a scalar or vector-valued function. If a function maps from :math:`R^n` to :math:`R^m`, its derivatives form an m-by-n matrix called the Jacobian, where an element :math:`(i, j)` is a partial derivative of f[i] with respect to ``xk[j]``. Parameters ---------- xk : array_like The coordinate vector at which to determine the gradient of `f`. f : callable Function of which to estimate the derivatives of. Has the signature ``f(xk, *args)`` where `xk` is the argument in the form of a 1-D array and `args` is a tuple of any additional fixed parameters needed to completely specify the function. The argument `xk` passed to this function is an ndarray of shape (n,) (never a scalar even if n=1). It must return a 1-D array_like of shape (m,) or a scalar. Suppose the callable has signature ``f0(x, *my_args, **my_kwargs)``, where ``my_args`` and ``my_kwargs`` are required positional and keyword arguments. Rather than passing ``f0`` as the callable, wrap it to accept only ``x``; e.g., pass ``fun=lambda x: f0(x, *my_args, **my_kwargs)`` as the callable, where ``my_args`` (tuple) and ``my_kwargs`` (dict) have been gathered before invoking this function. .. versionchanged:: 1.9.0 `f` is now able to return a 1-D array-like, with the :math:`(m, n)` Jacobian being estimated. epsilon : {float, array_like}, optional Increment to `xk` to use for determining the function gradient. If a scalar, uses the same finite difference delta for all partial derivatives. If an array, should contain one value per element of `xk`. Defaults to ``sqrt(np.finfo(float).eps)``, which is approximately 1.49e-08. \*args : args, optional Any other arguments that are to be passed to `f`. Returns ------- jac : ndarray The partial derivatives of `f` to `xk`. See Also -------- check_grad : Check correctness of gradient function against approx_fprime. Notes ----- The function gradient is determined by the forward finite difference formula:: f(xk[i] + epsilon[i]) - f(xk[i]) f'[i] = --------------------------------- epsilon[i] Examples -------- >>> import numpy as np >>> from scipy import optimize >>> def func(x, c0, c1): ... "Coordinate vector `x` should be an array of size two." ... return c0 * x[0]**2 + c1*x[1]**2 >>> x = np.ones(2) >>> c0, c1 = (1, 200) >>> eps = np.sqrt(np.finfo(float).eps) >>> optimize.approx_fprime(x, func, [eps, np.sqrt(200) * eps], c0, c1) array([ 2. , 400.00004208]) r)rlabs_steprHf0)rErrr%)xkrrrHr@s rArrs<T B B 2B Q9w"&2 //rCseed) position_numrF)r directionrngc F|}tj|}d}|dk(rtj||g|} | jdkDr t dt |} | j |j} ||| f|z} |} tjd}tj| | }n#|dk(r|} |} |}||g|}nt |dtjtjtj|t|| |g| z d zS) al Check the correctness of a gradient function by comparing it against a (forward) finite-difference approximation of the gradient. Parameters ---------- func : callable ``func(x0, *args)`` Function whose derivative is to be checked. grad : callable ``grad(x0, *args)`` Jacobian of `func`. x0 : ndarray Points to check `grad` against forward difference approximation of grad using `func`. args : \\*args, optional Extra arguments passed to `func` and `grad`. epsilon : float, optional Step size used for the finite difference approximation. It defaults to ``sqrt(np.finfo(float).eps)``, which is approximately 1.49e-08. direction : str, optional If set to ``'random'``, then gradients along a random vector are used to check `grad` against forward difference approximation using `func`. By default it is ``'all'``, in which case, all the one hot direction vectors are considered to check `grad`. If `func` is a vector valued function then only ``'all'`` can be used. rng : `numpy.random.Generator`, optional Pseudorandom number generator state. When `rng` is None, a new `numpy.random.Generator` is created using entropy from the operating system. Types other than `numpy.random.Generator` are passed to `numpy.random.default_rng` to instantiate a ``Generator``. The random numbers generated affect the random vector along which gradients are computed to check ``grad``. Note that `rng` is only used when `direction` argument is set to `'random'`. Returns ------- err : float The square root of the sum of squares (i.e., the 2-norm) of the difference between ``grad(x0, *args)`` and the finite difference approximation of `grad` using func at the points `x0`. See Also -------- approx_fprime Examples -------- >>> import numpy as np >>> def func(x): ... return x[0]**2 - 0.5 * x[1]**3 >>> def grad(x): ... return [2 * x[0], -1.5 * x[1]**2] >>> from scipy.optimize import check_grad >>> check_grad(func, grad, [1.5, -1.5]) 2.9802322387695312e-08 # may vary >>> rng = np.random.default_rng() >>> check_grad(func, grad, [1.5, -1.5], ... direction='random', seed=rng) 2.9802322387695312e-08 c ||||zzg|Sr;rT)wrrvrHs rAgzcheck_grad..g]sB1H$t$$rCrandomr!z1'random' can only be used with scalar valued func)rr!rFz1 is not a valid string for ``direction`` argumentr)rEr asanyarrayrrvr(standard_normalrrdotrrrr)rrrrrErFrHsteprK_gradrng_genrJ_args_funcvarsanalytical_grads rArrs/~ D BB%H d2oo. ::>%& &$S)  # #"(( # 4r1 $xx~&&* e r/D/I;'223 3 77266"&& =udCUC CaG rCcB||f|z}||||zzf|z}||z |z Sr;rT)rrfprimerrHf1f2s rAapprox_fhess_pr\ys; 2%$, B 2 >#d* ,B Gw rCc eZdZy)_LineSearchErrorNrrTrCrAr^r^rrCr^c |jdd}t|||||||fi|} | d#|!|| d|zz} || d| | d| dsd} | datj5tjdt i} dD]} | |vs|| | | <t |||||||fd|i| } ddd| d t| S#1swYxYw) a  Same as line_search_wolfe1, but fall back to line_search_wolfe2 if suitable step length is not found, and raise an exception if a suitable step length is not found. Raises ------ _LineSearchError If no suitable step size is found extra_conditionNrrr;ignore)c1c2r)popr"rcatch_warnings simplefilterr$r#r^) rrYrApkgfkold_fval old_old_fvalkwargsr`retxp1kwargs2keys rA_line_search_wolfe12rqs!jj!2D9O QB%| '% 'C 1vo93q6B;s1vsCFCF;C 1v~  $ $ & 0  ! !(,= >G+ /&=#)#;GCL /%QB%-|05D0(/0C 0 1v~  J 0 0s&&C CC h㈵>g?c |||| || | | ||d }t| } t||||fd| i|}|r+|d|d|d|d|d|d|d f}| r ||d fz }|S| r |d|d fS|dS) a Minimize a function using the BFGS algorithm. Parameters ---------- f : callable ``f(x,*args)`` Objective function to be minimized. x0 : ndarray Initial guess, shape (n,) fprime : callable ``f'(x,*args)``, optional Gradient of f. args : tuple, optional Extra arguments passed to f and fprime. gtol : float, optional Terminate successfully if gradient norm is less than `gtol` norm : float, optional Order of norm (Inf is max, -Inf is min) epsilon : int or ndarray, optional If `fprime` is approximated, use this value for the step size. callback : callable, optional An optional user-supplied function to call after each iteration. Called as ``callback(xk)``, where ``xk`` is the current parameter vector. maxiter : int, optional Maximum number of iterations to perform. full_output : bool, optional If True, return ``fopt``, ``func_calls``, ``grad_calls``, and ``warnflag`` in addition to ``xopt``. disp : bool, optional Print convergence message if True. retall : bool, optional Return a list of results at each iteration if True. xrtol : float, default: 0 Relative tolerance for `x`. Terminate successfully if step size is less than ``xk * xrtol`` where ``xk`` is the current parameter vector. c1 : float, default: 1e-4 Parameter for Armijo condition rule. c2 : float, default: 0.9 Parameter for curvature condition rule. hess_inv0 : None or ndarray, optional`` Initial inverse hessian estimate, shape (n, n). If None (default) then the identity matrix is used. Returns ------- xopt : ndarray Parameters which minimize f, i.e., ``f(xopt) == fopt``. fopt : float Minimum value. gopt : ndarray Value of gradient at minimum, f'(xopt), which should be near 0. Bopt : ndarray Value of 1/f''(xopt), i.e., the inverse Hessian matrix. func_calls : int Number of function_calls made. grad_calls : int Number of gradient calls made. warnflag : integer 1 : Maximum number of iterations exceeded. 2 : Gradient and/or function calls not changing. 3 : NaN result encountered. allvecs : list The value of `xopt` at each iteration. Only returned if `retall` is True. Notes ----- Optimize the function, `f`, whose gradient is given by `fprime` using the quasi-Newton method of Broyden, Fletcher, Goldfarb, and Shanno (BFGS). Parameters `c1` and `c2` must satisfy ``0 < c1 < c2 < 1``. See Also -------- minimize: Interface to minimization algorithms for multivariate functions. See ``method='BFGS'`` in particular. References ---------- Wright, and Nocedal 'Numerical Optimization', 1999, p. 198. Examples -------- >>> import numpy as np >>> from scipy.optimize import fmin_bfgs >>> def quadratic_cost(x, Q): ... return x @ Q @ x ... >>> x0 = np.array([-3, -4]) >>> cost_weight = np.diag([1., 10.]) >>> # Note that a trailing comma is necessary for a tuple with single element >>> fmin_bfgs(quadratic_cost, x0, args=(cost_weight,)) Optimization terminated successfully. Current function value: 0.000000 Iterations: 7 # may vary Function evaluations: 24 # may vary Gradient evaluations: 8 # may vary array([ 2.85169950e-06, -4.61820139e-07]) >>> def quadratic_cost_grad(x, Q): ... return 2 * Q @ x ... >>> fmin_bfgs(quadratic_cost, x0, quadratic_cost_grad, args=(cost_weight,)) Optimization terminated successfully. Current function value: 0.000000 Iterations: 7 Function evaluations: 8 Gradient evaluations: 8 array([ 2.85916637e-06, -4.54371951e-07]) ) gtolnormepsrr4rxrtolrcrd hess_inv0r^r?r<r=hess_invrnjevrr)rn_minimize_bfgs)rrrYrHrtrurr4rrrr^rwrcrdrxrr]rs rArrsj " $Dh'H Bf Hx H4 HCs8SZUS_v;F S]<  I) )G s8S^+ +s8OrCc t|t|| }t|j}|jdk(rd|_|t |dz}t|||||| |}|j}|j}||}||}d}t |}tj|t}||n|}|tjj|dz z}|}|r|g}d}t||} | |kDr||krtj || }! t#||||!|||d d | | \}"}#}$}}}%|"|!z}&||&z}'|rj'|'|'}|%||'}%|%|z }(|%}|d z }t)|| })t+||)rnUt||} | |krnA|"t|!z| | t|zzkrntj,|sd}ntj |(|&}*|*dk(rd}+| rd},t/d|,nd|*z }+||&ddtj0f|(tj0ddfz|+zz }-||(ddtj0f|&tj0ddfz|+zz }.tj |-tj ||.|+|&ddtj0fz|&tj0ddfzz}| |kDr||kr|}/|dk(r t2d},ns||k\r d }t2d},nbtj4| s8tj4|/s#tj4|j7r d}t2d},n t2d},| r\t/||,t9d|/dt9d|dt9d|j:dt9d|j<dt)|/|||j:|j<||dk(|,|| }0|r|0d<|0S#t$$rd}Y%wxYw) a Minimization of scalar function of one or more variables using the BFGS algorithm. Options ------- disp : bool Set to True to print convergence messages. maxiter : int Maximum number of iterations to perform. gtol : float Terminate successfully if gradient norm is less than `gtol`. norm : float Order of norm (Inf is max, -Inf is min). eps : float or ndarray If `jac is None` the absolute step size used for numerical approximation of the jacobian via forward differences. return_all : bool, optional Set to True to return a list of the best solution at each of the iterations. finite_diff_rel_step : None or array_like, optional If ``jac in ['2-point', '3-point', 'cs']`` the relative step size to use for numerical approximation of the jacobian. The absolute step size is computed as ``h = rel_step * sign(x) * max(1, abs(x))``, possibly adjusted to fit into the bounds. For ``jac='3-point'`` the sign of `h` is ignored. If None (default) then step is selected automatically. xrtol : float, default: 0 Relative tolerance for `x`. Terminate successfully if step size is less than ``xk * xrtol`` where ``xk`` is the current parameter vector. c1 : float, default: 1e-4 Parameter for Armijo condition rule. c2 : float, default: 0.9 Parameter for curvature condition rule. hess_inv0 : None or ndarray, optional Initial inverse hessian estimate, shape (n, n). If None (default) then the identity matrix is used. workers : int, map-like callable, optional A map-like callable, such as `multiprocessing.Pool.map` for evaluating any numerical differentiation in parallel. This evaluation is carried out as ``workers(fun, iterable)``. .. versionadded:: 1.16.0 Notes ----- Parameters `c1` and `c2` must satisfy ``0 < c1 < c2 < 1``. If minimization doesn't complete successfully, with an error message of ``Desired error not necessarily achieved due to precision loss``, then consider setting `gtol` to a higher value. This precision loss typically occurs when the (finite difference) numerical differentiation cannot provide sufficient precision to satisfy the `gtol` termination criterion. This can happen when working in single precision and a callable jac is not provided. For single precision problems a `gtol` of 1e-3 seems to work. rrMNr)rHrrrrrr0.++}Ô%IT)rrrcrdr!rg@@z.Divide-by-zero encountered: rhok assumed largeTrr5r4rr6r2rrrrr Gradient evaluations: ) r<r=ryrrzrr2rr?rr)rrxrrrrrrr<rrErintlinalgrurrPrqr^rrr*r_print_success_message_or_warnnewaxisrisnanrrrngev)1r<rrHr=r^rtrurvr4rrrrwrcrdrxrrrrrmyfprimerjrir(r&IrwrkrArr<gnormrhalpha_kfcgcgfkp1skxkp1ykrZrhok_invrhokrA1A2r;r=s1 rAr{r{Asz?+Y' F    B ww!|b'C- !#r3T37K*1 3B AwwHuH 2,C A BA qAYBbiinnS1A55L B $H CT "E 4 > {{8$H 66"b> r>DF.tS9=D ArzzM"R A %66= = ArzzM"R A %66= = VVBr2 '4"Q ]2C+C13BJJM1B,CDu 4.>.@e$i( &x5 1$q:; %aU+, /{;< /{;< #!#%-]SB !#F#y Ma  H  sO88 PPc`|s t|ytj||xstdy)Nrr{)rrrr)r<rwarntypes rArrs!  g gx:?qIrCg?c|||| || d}t| } t||||f| | | d|}|r#|d|d|d|d|df}| r ||dfz }|S| r |d|dfS|dS) a: Minimize a function using a nonlinear conjugate gradient algorithm. Parameters ---------- f : callable, ``f(x, *args)`` Objective function to be minimized. Here `x` must be a 1-D array of the variables that are to be changed in the search for a minimum, and `args` are the other (fixed) parameters of `f`. x0 : ndarray A user-supplied initial estimate of `xopt`, the optimal value of `x`. It must be a 1-D array of values. fprime : callable, ``fprime(x, *args)``, optional A function that returns the gradient of `f` at `x`. Here `x` and `args` are as described above for `f`. The returned value must be a 1-D array. Defaults to None, in which case the gradient is approximated numerically (see `epsilon`, below). args : tuple, optional Parameter values passed to `f` and `fprime`. Must be supplied whenever additional fixed parameters are needed to completely specify the functions `f` and `fprime`. gtol : float, optional Stop when the norm of the gradient is less than `gtol`. norm : float, optional Order to use for the norm of the gradient (``-np.inf`` is min, ``np.inf`` is max). epsilon : float or ndarray, optional Step size(s) to use when `fprime` is approximated numerically. Can be a scalar or a 1-D array. Defaults to ``sqrt(eps)``, with eps the floating point machine precision. Usually ``sqrt(eps)`` is about 1.5e-8. maxiter : int, optional Maximum number of iterations to perform. Default is ``200 * len(x0)``. full_output : bool, optional If True, return `fopt`, `func_calls`, `grad_calls`, and `warnflag` in addition to `xopt`. See the Returns section below for additional information on optional return values. disp : bool, optional If True, return a convergence message, followed by `xopt`. retall : bool, optional If True, add to the returned values the results of each iteration. callback : callable, optional An optional user-supplied function, called after each iteration. Called as ``callback(xk)``, where ``xk`` is the current value of `x0`. c1 : float, default: 1e-4 Parameter for Armijo condition rule. c2 : float, default: 0.4 Parameter for curvature condition rule. Returns ------- xopt : ndarray Parameters which minimize f, i.e., ``f(xopt) == fopt``. fopt : float, optional Minimum value found, f(xopt). Only returned if `full_output` is True. func_calls : int, optional The number of function_calls made. Only returned if `full_output` is True. grad_calls : int, optional The number of gradient calls made. Only returned if `full_output` is True. warnflag : int, optional Integer value with warning status, only returned if `full_output` is True. 0 : Success. 1 : The maximum number of iterations was exceeded. 2 : Gradient and/or function calls were not changing. May indicate that precision was lost, i.e., the routine did not converge. 3 : NaN result encountered. allvecs : list of ndarray, optional List of arrays, containing the results at each iteration. Only returned if `retall` is True. See Also -------- minimize : common interface to all `scipy.optimize` algorithms for unconstrained and constrained minimization of multivariate functions. It provides an alternative way to call ``fmin_cg``, by specifying ``method='CG'``. Notes ----- This conjugate gradient algorithm is based on that of Polak and Ribiere [1]_. Conjugate gradient methods tend to work better when: 1. `f` has a unique global minimizing point, and no local minima or other stationary points, 2. `f` is, at least locally, reasonably well approximated by a quadratic function of the variables, 3. `f` is continuous and has a continuous gradient, 4. `fprime` is not too large, e.g., has a norm less than 1000, 5. The initial guess, `x0`, is reasonably close to `f` 's global minimizing point, `xopt`. Parameters `c1` and `c2` must satisfy ``0 < c1 < c2 < 1``. References ---------- .. [1] Wright & Nocedal, "Numerical Optimization", 1999, pp. 120-122. Examples -------- Example 1: seek the minimum value of the expression ``a*u**2 + b*u*v + c*v**2 + d*u + e*v + f`` for given values of the parameters and an initial guess ``(u, v) = (0, 0)``. >>> import numpy as np >>> args = (2, 3, 7, 8, 9, 10) # parameter values >>> def f(x, *args): ... u, v = x ... a, b, c, d, e, f = args ... return a*u**2 + b*u*v + c*v**2 + d*u + e*v + f >>> def gradf(x, *args): ... u, v = x ... a, b, c, d, e, f = args ... gu = 2*a*u + b*v + d # u-component of the gradient ... gv = b*u + 2*c*v + e # v-component of the gradient ... return np.asarray((gu, gv)) >>> x0 = np.asarray((0, 0)) # Initial guess. >>> from scipy import optimize >>> res1 = optimize.fmin_cg(f, x0, fprime=gradf, args=args) Optimization terminated successfully. Current function value: 1.617021 Iterations: 4 Function evaluations: 8 Gradient evaluations: 8 >>> res1 array([-1.80851064, -0.25531915]) Example 2: solve the same problem using the `minimize` function. (This `myopts` dictionary shows all of the available options, although in practice only non-default values would be needed. The returned value will be a dictionary.) >>> opts = {'maxiter' : None, # default value. ... 'disp' : True, # non-default value. ... 'gtol' : 1e-5, # default value. ... 'norm' : np.inf, # default value. ... 'eps' : 1.4901161193847656e-08} # default value. >>> res2 = optimize.minimize(f, x0, jac=gradf, args=args, ... method='CG', options=opts) Optimization terminated successfully. Current function value: 1.617021 Iterations: 4 Function evaluations: 8 Gradient evaluations: 8 >>> res2.x # minimum found array([-1.80851064, -0.25531915]) )rtrurvrr4rr^rcrdr?r<rrzrr)rn _minimize_cg)rrrYrHrtrurr4rrrr^rcrdrr]rs rArrs@  "Dh'H q"dF X"  Cc(CJF S[#h-O  I) )G s8S^+ +s8OrCc ."#$%&'()t|| }t|j}|t|dz}t |||||| |}|j }|j %||}%|$d}|)|tjj$dz z}|r)g}d}$ &t$}d(|kDr||krtj$$#dg"d#$%&)fd '"'(fd } t|%)&$||| | d d | \}}}}}}|"dk(r "\})&$}n'||\})&$}|rj)|d z }t)|}t!||rn |kDr||kr|}|dk(r t"d} ns||k\r d }t"d} nbtj$|s8tj$|s#tj$)j'r d}t"d} n t"d} | r\t)|| t+d|dt+d|dt+d|j,dt+d|j.dt|$|j,|j.||dk(| )| }!|r|!d<|!S#t$rd}Y$wxYw)a Minimization of scalar function of one or more variables using the conjugate gradient algorithm. Options ------- disp : bool Set to True to print convergence messages. maxiter : int Maximum number of iterations to perform. gtol : float Gradient norm must be less than `gtol` before successful termination. norm : float Order of norm (Inf is max, -Inf is min). eps : float or ndarray If `jac is None` the absolute step size used for numerical approximation of the jacobian via forward differences. return_all : bool, optional Set to True to return a list of the best solution at each of the iterations. finite_diff_rel_step : None or array_like, optional If ``jac in ['2-point', '3-point', 'cs']`` the relative step size to use for numerical approximation of the jacobian. The absolute step size is computed as ``h = rel_step * sign(x) * max(1, abs(x))``, possibly adjusted to fit into the bounds. For ``jac='3-point'`` the sign of `h` is ignored. If None (default) then step is selected automatically. c1 : float, default: 1e-4 Parameter for Armijo condition rule. c2 : float, default: 0.4 Parameter for curvature condition rule. workers : int, map-like callable, optional A map-like callable, such as `multiprocessing.Pool.map` for evaluating any numerical differentiation in parallel. This evaluation is carried out as ``workers(fun, iterable)``. .. versionadded:: 1.16.0 Notes ----- Parameters `c1` and `c2` must satisfy ``0 < c1 < c2 < 1``. Nr)r=rHrrrrrr}g{Gz?c | zz}| |}|z }tdtj||z }| | zz}t| }|||||fS)Nrr})rrErPr) alpharrrbeta_kpkp1rdeltakrirrurhrAs rApolak_ribiere_powell_stepz/_minimize_cg..polak_ribiere_powell_step sp ?D} BBFF2u-67F6FRK'DEt,E4ue4 4rCc ||dd\}}}}}| krytj|| tj||zkS)NT)rErP) rrfp1rrArhrir cached_steprtrsigma_3s rAdescent_conditionz'_minimize_cg..descent_conditionsZ7ueDKN(3 %E2r3}66"c?whS1A&AA ArCr~r)rcrdrrr`r!rr5r4rr6r2rrrrrr) r<r=rrzrr2rr?rrr;)rrrrrr<rrErrurrPrqr^rrr*rrrrrrr)*r<rrHr=r^rtrurvr4rrrrcrdrrrrrrjr(rkrr<rrrrrrrZr;rr=rrrirrhrrrAs* `` @@@@@@@@rArrs`?+ F    Bb'C- !#rss7K*1 3B AwwHuH 2,C A BbiinnS1A55L $H B CT "EG 4.>.@e$i( &x5 1$q:; %aU+, /{;< /{;< #BGG!#%-]SB !#F#y MW  H  s4J JJc|||| | d}t| } t||||||f| | |d|}| r'|d|d|d|d|d|df}| r ||d fz }|S| r |d|d fS|dS) a Unconstrained minimization of a function using the Newton-CG method. Parameters ---------- f : callable ``f(x, *args)`` Objective function to be minimized. x0 : ndarray Initial guess. fprime : callable ``f'(x, *args)`` Gradient of f. fhess_p : callable ``fhess_p(x, p, *args)``, optional Function which computes the Hessian of f times an arbitrary vector, p. fhess : callable ``fhess(x, *args)``, optional Function to compute the Hessian matrix of f. args : tuple, optional Extra arguments passed to f, fprime, fhess_p, and fhess (the same set of extra arguments is supplied to all of these functions). epsilon : float or ndarray, optional If fhess is approximated, use this value for the step size. callback : callable, optional An optional user-supplied function which is called after each iteration. Called as callback(xk), where xk is the current parameter vector. avextol : float, optional Convergence is assumed when the average relative error in the minimizer falls below this amount. maxiter : int, optional Maximum number of iterations to perform. full_output : bool, optional If True, return the optional outputs. disp : bool, optional If True, print convergence message. retall : bool, optional If True, return a list of results at each iteration. c1 : float, default: 1e-4 Parameter for Armijo condition rule. c2 : float, default: 0.9 Parameter for curvature condition rule Returns ------- xopt : ndarray Parameters which minimize f, i.e., ``f(xopt) == fopt``. fopt : float Value of the function at xopt, i.e., ``fopt = f(xopt)``. fcalls : int Number of function calls made. gcalls : int Number of gradient calls made. hcalls : int Number of Hessian calls made. warnflag : int Warnings generated by the algorithm. 1 : Maximum number of iterations exceeded. 2 : Line search failure (precision loss). 3 : NaN result encountered. allvecs : list The result at each iteration, if retall is True (see below). See also -------- minimize: Interface to minimization algorithms for multivariate functions. See the 'Newton-CG' `method` in particular. Notes ----- Only one of `fhess_p` or `fhess` need to be given. If `fhess` is provided, then `fhess_p` will be ignored. If neither `fhess` nor `fhess_p` is provided, then the hessian product will be approximated using finite differences on `fprime`. `fhess_p` must compute the hessian times an arbitrary vector. If it is not given, finite-differences on `fprime` are used to compute it. Newton-CG methods are also called truncated Newton methods. This function differs from scipy.optimize.fmin_tnc because 1. scipy.optimize.fmin_ncg is written purely in Python using NumPy and scipy while scipy.optimize.fmin_tnc calls a C function. 2. scipy.optimize.fmin_ncg is only for unconstrained minimization while scipy.optimize.fmin_tnc is for unconstrained minimization or box constrained minimization. (Box constraints give lower and upper bounds for each variable separately.) Parameters `c1` and `c2` must satisfy ``0 < c1 < c2 < 1``. References ---------- Wright & Nocedal, 'Numerical Optimization', 1999, p. 140. )rrvr4rrrr?r<rrznhevrr)rn_minimize_newtoncg)rrrYfhess_pfhessrHavextolrr4rrrr^rcrdrr]rs rArrYsB  "D h'H QD&% F&.2" F@D FCs8SZVc&kv;H /  I) )G s8S^+ +s8OrCc l 56789:;<t|| td|}|}|}|}| :t|j}t |||||||;;j };j };j|}|tvst|tr d};fd}|}5 6789:;|#|!krn|!|tA<|$||})n!|<|$g|})7d z 7n'j7|$})t|)jC})tj6|$|)}*d|*cxkrd|zkrnnne|*dkr|%dkDrnY|&|* z |z}"nN|&|*z }+|"|+|$zz }"|#|+|)zz }#tj6|#|#},|,|&z }-|# |-|$zz}$|%d z }%|,}& d}|d|S|"}.| 6 tE||<|.69|| |  \}/}0}19}}2|/|.z}3<|3z <:r5jI<8d z 8tK<9}4tM||4r |ddStj(j+|3d }||kDrltjN9stjN|r|dt&dSt&d}|d|S#tF$rd t&dz}|d|cYSwxYw)a Minimization of scalar function of one or more variables using the Newton-CG algorithm. Note that the `jac` parameter (Jacobian) is required. Options ------- disp : bool Set to True to print convergence messages. xtol : float Average relative error in solution `xopt` acceptable for convergence. maxiter : int Maximum number of iterations to perform. eps : float or ndarray If `hessp` is approximated, use this value for the step size. return_all : bool, optional Set to True to return a list of the best solution at each of the iterations. c1 : float, default: 1e-4 Parameter for Armijo condition rule. c2 : float, default: 0.9 Parameter for curvature condition rule. workers : int, map-like callable, optional A map-like callable, such as `multiprocessing.Pool.map` for evaluating any numerical differentiation in parallel. This evaluation is carried out as ``workers(fun, iterable)``. .. versionadded:: 1.16.0 Notes ----- Parameters `c1` and `c2` must satisfy ``0 < c1 < c2 < 1``. Nz)Jacobian is required for Newton-CG method)rHrrrcDj|j|Sr;)rrP)r?rrHrs rA_hesspz"_minimize_newtoncg.._hessps771:>>!$ $rCc L rkt||td dtddtd jdtd jdtdd }t | j j||dk(|  } r|d <|S) Nrrrrrrz Hessian evaluations: r) r<r=rrzrrr2rr?rr)rrrrr) r<rr;r=rrrihcallsr(rjrrrAs rA terminatez%_minimize_newtoncg..terminates  *8S 9 5hq\B C )!A/ 0 3BGGA;? @ 3BGGA;? @ 26!*= >Dc%'WW6()1Q$%'  'F9  rCrrz Warning: r4r!r}rrrzMWarning: CG iterations didn't converge. The Hessian is not positive definite.)rcrdr5rrrar6r2)(rrvrrrr<rrr- isinstancer rrEfinforrrGrrvrrrurmathrrrrPr rrrr\squeezerqr^rrr*r)=r<rrHr=rhesspr^rrvr4rrrcrdrrrrrrrrY_hrr cg_maxiter update_l1normrk float64epsrbmaggradetatermcondxsupiripsupiidri0rtk2Apcurvalphaidri1betairhalphakrrrupdaterZrrirr(rjrrrAs= ` @@@@@@@@rArrscN?+ {DEEG EGG F    B ! R44 B A WWF B j^< %"Fb'#+CGJ r7W DHHUO''M B $ A CuHL"**%))J $  < ::CQ$ $BZK))...*#tyy)*=c"gRXX.R vvb"~   A aKF #% %Bvv}}RVVBZ(H4}?'E67CB U2T2BaKFUU5\$$&B66%$DD*A N*E!TENQ.ED[F Ve^ #E &2+ B66"b>D4KEC%%-'E FADA% %F(CQ$ $ b %)!VRS*2LRBP :FBHlE" f   NN2  Q,rx@ $X/B CQ# # v15 _ $ d 88H -!8Q 67 7i(C  )  % ::CQ$ $ %s?PP32P3c`|||d}t|||f|fi|} |r| d| d| d| dfS| dS)a Bounded minimization for scalar functions. Parameters ---------- func : callable f(x,*args) Objective function to be minimized (must accept and return scalars). x1, x2 : float or array scalar Finite optimization bounds. args : tuple, optional Extra arguments passed to function. xtol : float, optional The convergence tolerance. maxfun : int, optional Maximum number of function evaluations allowed. full_output : bool, optional If True, return optional outputs. disp: int, optional If non-zero, print messages. ``0`` : no message printing. ``1`` : non-convergence notification messages only. ``2`` : print a message on convergence too. ``3`` : print iteration results. Returns ------- xopt : ndarray Parameters (over given interval) which minimize the objective function. fval : number (Optional output) The function value evaluated at the minimizer. ierr : int (Optional output) An error flag (0 if converged, 1 if maximum number of function calls reached). numfunc : int (Optional output) The number of function calls made. See also -------- minimize_scalar: Interface to minimization algorithms for scalar univariate functions. See the 'Bounded' `method` in particular. Notes ----- Finds a local minimizer of the scalar function `func` in the interval x1 < xopt < x2 using Brent's method. (See `brent` for auto-bracketing.) References ---------- .. [1] Forsythe, G.E., M. A. Malcolm, and C. B. Moler. "Computer Methods for Mathematical Computations." Prentice-Hall Series in Automatic Computation 259 (1977). .. [2] Brent, Richard P. Algorithms for Minimization Without Derivatives. Courier Corporation, 2013. Examples -------- `fminbound` finds the minimizer of the function in the given range. The following examples illustrate this. >>> from scipy import optimize >>> def f(x): ... return (x-1)**2 >>> minimizer = optimize.fminbound(f, -4, 4) >>> minimizer 1.0 >>> minimum = f(minimizer) >>> minimum 0.0 >>> res = optimize.fminbound(f, 3, 4, full_output=True) >>> minimizer, fval, ierr, numfunc = res >>> minimizer 3.000005960860986 >>> minimum = f(minimizer) >>> minimum, fval (4.000023843479476, 4.000023843479476) )rr4rr?r<rr)_minimize_scalar_bounded) rx1x2rHrrrroptionsr]s rArrsYf G #4"b4 C7 CC3xUS]CK??3xrCc t||}t|dk7r td|\}} t|r t| s td|| kDr tdd} d} d} t d} d d t d z z}|| }}||||z zz}||}}d x}}|}||g|}d }d ||f}t j }|x}}d ||zz}| t j|z|d z z}d|z}|dkDr(tdt| td|| fzzt j||z |d ||z zz kDr-d } t j||kDrd} ||z ||z z}!||z ||z z}"||z |"z||z |!zz }#d|"|!z z}"|"d kDr|# }#t j|"}"|}!|}t j|#t jd |"z|!zkr\|#|"||z zkDrQ|#|"||z zkrF|#d z|"z }||z}d} ||z |ks||z |kr)t j||z ||z dk(z}$||$z}nd } | r||k\r||z }n||z }||z}d} t j||dk(z}$||$t jt j||zz}||g|}|d z }|||f}|dkDrtd|| fzz||kr||k\r|}n|}||}}||}}||}}n0||kr|}n|}||ks||k(r ||}}||}}n||ks ||k(s||k(r||}}d ||zz}| t j|z|d z z}d|z}||k\rd } n&t j||z |d ||z zz kDr-t j|s*t j|st j|rd} |}%|dkDrt|| |%|||t|%| | dk(ddtddj| d|||}&|&S)a Options ------- maxiter : int Maximum number of iterations to perform. disp: int, optional If non-zero, print messages. ``0`` : no message printing. ``1`` : non-convergence notification messages only. ``2`` : print a message on convergence too. ``3`` : print iteration results. xatol : float Absolute error in solution `xopt` acceptable for convergence. rzbounds must have two elements.z+Optimization bounds must be finite scalars.z(The lower bound exceeds the upper bound.rz2 Func-count x f(x) Procedurez initialgOd@a"f%3w!mH)UdNQ$b)b2g!^%@ADD"  s 5F MrCc4eZdZ ddZddZdZdZd dZy) Brentc||_||_||_||_d|_d|_d|_d|_d|_d|_ ||_ y)Ngdy=gŋ!r?r) rrHtolr4_mintol_cgxminr;iterfuncallsr)r@rrHrr4rrs rArBzBrent.__init__ sO         rCNc||_yr;)brack)r@rs rA set_bracketzBrent.set_bracket s  rCc|j}|j}|j}|t||\}}}}}} } nt |dk(rt||d|d|\}}}}}} } npt |dk(rW|\}}}||kDr||}}||kr||ks t d||f|z}||f|z}||f|z} ||kr|| ks t dd} n t d |||||| | fS) NrHrrr!xaxbrHrWBracketing values (xa, xb, xc) do not fulfill this requirement: (xa < xb) and (xb < xc)cBracketing values (xa, xb, xc) do not fulfill this requirement: (f(xb) < f(xa)) and (f(xb) < f(xc))3Bracketing interval must be length 2 or 3 sequence.)rrHrr rrv) r@rrHrrrr6fafbrrs rAget_bracket_infozBrent.get_bracket_info s=yyyy  =/6t$/G ,BBBH Z1_/6ta:?(0O ,BBBH Z1_JBBRRB"W27 I &B &B &B"W27 M H78 82r2r2x//rCc|j}|j\}}}}}}}|j} |j} |x} x} } |x}x}}||kr|}|}n|}|}d}d}|jdkDr7t dt dddddddd t |d d| d d|d ||j kr+|jtj| z| z}d |z}d ||zz}tj| |z |d ||z zz krntj||kr| |k\r|| z }n|| z }| |z}n| | z ||z z}| | z ||z z}| | z |z| | z |zz }d ||z z}|dkDr| }tj|}|}}|||| z zkDrh|||| z zkr]tj|tjd |z|zkr,|dz|z }| |z}||z |ks||z |kr$|| z dk\r|}n| }n| |k\r|| z }n|| z }| |z}tj||kr|dk\r| |z}n | |z }n| |z}||f|jz}|dz }||kDr1|| kr|}n|}||ks| | k(r | } |} |}|}n*||ks | | k(s| | k(r|} |}n|| k\r| }n| }| } | } |} |}|}|}|jdkDrt |d d| d d|d |dz }||j kr+| |_ ||_ ||_ ||_y)Nrrrr Func-count^12r?f(x) ^12^12g^12.6grrrr!)rrrrrrr4rrErrHrr;rr)r@rrrr6rrrrrrr?rIrJfwfvrrrdeltaxrrrxmidrtmp1tmp2rdx_tempurs rAoptimizezBrent.optimize syy+/+@+@+B(BBB,,hhAR" GAAAA 99q= #J \#&aCy&? @ XdO1QvJa6{; <dll"88bffQi''1D:D!a%=Dvva$h4#Q-#78v$&IUFUFFlA"r'*A"r'*UdNa!et^3dTk*3JAvvd| Q'a$!a%..@RVVC$J,@%AAc'D.CCAQ$1q5D.!8q="&C#'%CT !"Q!"Q,Cs d"!8DADAGtyy(*B MHREAA"H!q&AABBBhAFQABFAAyy1}$q6 !Bv;?@ AIDcdll"l     rCcz|r.|j|j|j|jfS|jSr;)rr;rr)r@rs rA get_resultzBrent.get_result5 s. 99diiDMMA A99 rC)rT`sbO>rrr;)F)rPrQrRrBrrrrrTrCrArr s$;>%& $0Lr!hrCrcZ||d}t|||fi|}|r|d|d|d|dfS|dS)a Given a function of one variable and a possible bracket, return a local minimizer of the function isolated to a fractional precision of tol. Parameters ---------- func : callable f(x,*args) Objective function. args : tuple, optional Additional arguments (if present). brack : tuple, optional Either a triple ``(xa, xb, xc)`` satisfying ``xa < xb < xc`` and ``func(xb) < func(xa) and func(xb) < func(xc)``, or a pair ``(xa, xb)`` to be used as initial points for a downhill bracket search (see `scipy.optimize.bracket`). The minimizer ``x`` will not necessarily satisfy ``xa <= x <= xb``. tol : float, optional Relative error in solution `xopt` acceptable for convergence. full_output : bool, optional If True, return all output args (xmin, fval, iter, funcalls). maxiter : int, optional Maximum number of iterations in solution. Returns ------- xmin : ndarray Optimum point. fval : float (Optional output) Optimum function value. iter : int (Optional output) Number of iterations. funcalls : int (Optional output) Number of objective function evaluations made. See also -------- minimize_scalar: Interface to minimization algorithms for scalar univariate functions. See the 'Brent' `method` in particular. Notes ----- Uses inverse parabolic interpolation when possible to speed up convergence of golden section method. Does not ensure that the minimum lies in the range specified by `brack`. See `scipy.optimize.fminbound`. Examples -------- We illustrate the behaviour of the function when `brack` is of size 2 and 3 respectively. In the case where `brack` is of the form ``(xa, xb)``, we can see for the given values, the output does not necessarily lie in the range ``(xa, xb)``. >>> def f(x): ... return (x-1)**2 >>> from scipy import optimize >>> minimizer = optimize.brent(f, brack=(1, 2)) >>> minimizer 1 >>> res = optimize.brent(f, brack=(-1, 0.5, 2), full_output=True) >>> xmin, fval, iter, funcalls = res >>> f(xmin), fval (0.0, 0.0) rr4r?r<rr)_minimize_scalar_brentrrHrrrr4rr]s rArr< sPN!#G ud >g >C3xUSZV<<3xrCc t||}|dkrtd|t|||d||}|j||j |j d\} } } } | |kxr-t j| xst j|  } | rd|d}n;| |k\rd}t j| st j| r td }|r t| t| | | | |  S) a: Options ------- maxiter : int Maximum number of iterations to perform. xtol : float Relative error in solution `xopt` acceptable for convergence. disp : int, optional If non-zero, print messages. ``0`` : no message printing. ``1`` : non-convergence notification messages only. ``2`` : print a message on convergence too. ``3`` : print iteration results. Notes ----- Uses inverse parabolic interpolation when possible to speed up convergence of golden section method. rztolerance should be >= 0, got T)rrHrrr4r)rk Optimization terminated successfully; The returned value satisfies the termination criteria (using xtol =  )& Maximum number of iterations exceededr6)r<r?rrr2r) rrvrrrrrErrrr)rrrHrr4rrrrr?r;rrr2rs rArr s 6?+ C Qw9#ABB t$C"G$ @E e NN))d);AtS$GmCRXXa[%BBHHTN CG$$(6- '>?G 88A;"((4.(/0G &7{G< daSt")7 <>> def f(x): ... return (x-1)**2 >>> from scipy import optimize >>> minimizer = optimize.golden(f, brack=(1, 2)) >>> minimizer 1 >>> res = optimize.golden(f, brack=(-1, 0.5, 2), full_output=True) >>> xmin, fval, funcalls = res >>> f(xmin), fval (9.925165290385052e-18, 9.925165290385052e-18) rr?r<r)_minimize_scalar_goldenrs rAr r sGJw/G !$t ?w ?C3xUS[003xrCc t||}|t||\}} } } } } }nt|dk(rt||d|d|\}} } } } } }npt|dk(rW|\}} } || kDr|| }} || kr| | ks td||f|z} || f|z} || f|z} | | kr| | ks tdd}n td d }d |z }| }|}t j | | z t j | |z kDr| }| || | z zz}n | }| || |z zz }||f|z}||f|z}|dz }d}|dkDr!t d t d dd ddd ddt|D]}t j ||z |t j |t j |zzkrno||kr|}|}||z||zz}|}||f|z}n|}|}||z||zz}|}||f|z}|dz }|dkDr$||kr||}}n||}}t |dd |dd |d|dz }||kr|}|}n|}|}||kxr-t j|xst j| }|rd|d}n;||k\rd}t j|st j|r td}|r t| t|||||S)a Options ------- xtol : float Relative error in solution `xopt` acceptable for convergence. maxiter : int Maximum number of iterations to perform. disp: int, optional If non-zero, print messages. ``0`` : no message printing. ``1`` : non-convergence notification messages only. ``2`` : print a message on convergence too. ``3`` : print iteration results. rrrr!rrrrrgz7?rrrrr?rrrr rrrr6r<rr?rr2r) rr rrvrErrr rrrr)rrrHrr4rrrrrr6rrrr_gR_gCx3rrrrZr[rrrr;r2rs rAr r  s*?+ C }+24d+C(BBB Uq+24E!H6;AhT,K(BBB Uq B GBbrBwE REDL " REDL " REDL "brBwI NOO C )C B B rBw"&&b/)  #b/ !  #b/ !  B  B MH C ax c  c"!C9AfT];< 7^ 66"r'?cRVVBZ"&&*%<= =  GBBrC"H$BB &BBBrC"H$BB &BA  !8Rdd XdO1T&M4-@ A q/4 RGmFRXXd^%Erxx~ FG$$(6- '>?G 88D>RXXd^(/0G &7{G< dTs")7 <>> import numpy as np >>> import matplotlib.pyplot as plt >>> from scipy.optimize import bracket >>> def f(x): ... return 10*x**2 + 3*x + 5 >>> x = np.linspace(-2, 2) >>> y = f(x) >>> init_xa, init_xb = 0.1, 1 >>> xa, xb, xc, fa, fb, fc, funcalls = bracket(f, xa=init_xa, xb=init_xb) >>> plt.axvline(x=init_xa, color="k", linestyle="--") >>> plt.axvline(x=init_xb, color="k", linestyle="--") >>> plt.plot(x, y, "-k") >>> plt.plot(xa, fa, "bx") >>> plt.plot(xb, fb, "rx") >>> plt.plot(xc, fc, "bx") >>> plt.show() Note that both initial points were to the right of the minimum, and the third point was found in the "downhill" direction: the direction in which the function appeared to be decreasing (to the left). The final points are strictly ordered, and the function value at the middle point is less than the function values at the endpoints; it follows that a minimum must lie within the bracket. gw?gO ;rrrzNo valid bracket was found before the iteration limit was reached. Consider trying different initial points or increasing `maxiter`.r!rzcThe algorithm terminated without finding a valid bracket. Consider trying different initial points.)rErr RuntimeErrorr BracketErrordata)rrrrH grow_limitr4_gold_verysmall_numrrr6rrrrrvaldenomrIwlimrr cond1cond2cond3rs rAr r swT EN ZZR !FB ut| B ut| B RRBRB erBw B  BH D 7RBG$RBG$Tk 66#; '.(E#IE 27d"b2g%55> >J"r'**' '>s# #   FrAv  $t %B MHRr'Ub2g&&At %B MH$h #s *At %B MH$ha 3 &t %B MHR"r'**QD4K)A Ub2g&&At %B MH      q 7v"W !r .myfunc sAbL!!rCN)rrr!)r;rd)rc:tj|Sr;)rEtan)r?rIs rAz$_linesearch_powell.. s&+rC)rErr9rtupler?r<rEisneginfisposinf_linesearch_powellrarctanrL) rrrHrr$r%r;r] alpha_minfretboundrIs ``` @rArQrQ s*" 66":!%!1q" HQB7GH  !4)*@*0$cK%%4 ^QVR B [A ;;uQx R[[q%:%dAr#F FU1X&r{{58/D*65c JCB77AFB& & IIeAh'58)<>> def f(x): ... return x**2 >>> from scipy import optimize >>> minimum = optimize.fmin_powell(f, -1) Optimization terminated successfully. Current function value: 0.000000 Iterations: 2 Function evaluations: 16 >>> minimum array(0.0) )rrr4r3rdirecrr^r?r<rWrrrr)rn_minimize_powell)rrrHrrr4rrrrr^rWrr]rs rArr sl  "Dh'H 4T EH E ECs8SZWs5zv;H /  I) )G s8S^+ +s8OrCc t| |} | }t|j}|r|g}t|}| | |dz}|dz} nW|*| tj k(r|dz}ntAd|dtAd|dtAd|ddt-|| ||d|%|%dk(|&|}'|r|'d<|'S#t8$rY"wxYw)a Minimization of scalar function of one or more variables using the modified Powell algorithm. Parameters ---------- fun : callable The objective function to be minimized:: fun(x, *args) -> float where ``x`` is a 1-D array with shape (n,) and ``args`` is a tuple of the fixed parameters needed to completely specify the function. x0 : ndarray, shape (n,) Initial guess. Array of real elements of size (n,), where ``n`` is the number of independent variables. args : tuple, optional Extra arguments passed to the objective function and its derivatives (`fun`, `jac` and `hess` functions). method : str or callable, optional The present documentation is specific to ``method='powell'``, but other options are available. See documentation for `scipy.optimize.minimize`. bounds : sequence or `Bounds`, optional Bounds on decision variables. There are two ways to specify the bounds: 1. Instance of `Bounds` class. 2. Sequence of ``(min, max)`` pairs for each element in `x`. None is used to specify no bound. If bounds are not provided, then an unbounded line search will be used. If bounds are provided and the initial guess is within the bounds, then every function evaluation throughout the minimization procedure will be within the bounds. If bounds are provided, the initial guess is outside the bounds, and `direc` is full rank (or left to default), then some function evaluations during the first iteration may be outside the bounds, but every function evaluation after the first iteration will be within the bounds. If `direc` is not full rank, then some parameters may not be optimized and the solution is not guaranteed to be within the bounds. options : dict, optional A dictionary of solver options. All methods accept the following generic options: maxiter : int Maximum number of iterations to perform. Depending on the method each iteration may use several function evaluations. disp : bool Set to True to print convergence messages. See method-specific options for ``method='powell'`` below. callback : callable, optional Called after each iteration. The signature is:: callback(xk) where ``xk`` is the current parameter vector. Returns ------- res : OptimizeResult The optimization result represented as a ``OptimizeResult`` object. Important attributes are: ``x`` the solution array, ``success`` a Boolean flag indicating if the optimizer exited successfully and ``message`` which describes the cause of the termination. See `OptimizeResult` for a description of other attributes. Options ------- disp : bool Set to True to print convergence messages. xtol : float Relative error in solution `xopt` acceptable for convergence. ftol : float Relative error in ``fun(xopt)`` acceptable for convergence. maxiter, maxfev : int Maximum allowed number of iterations and function evaluations. Will default to ``N*1000``, where ``N`` is the number of variables, if neither `maxiter` or `maxfev` is set. If both `maxiter` and `maxfev` are set, minimization will stop at the first reached. direc : ndarray Initial set of direction vectors for the Powell method. return_all : bool, optional Set to True to return a list of the best solution at each of the iterations. rrzBdirec input is not full rank, some parameters may not be optimizedrr{)NNrrrJ)rr$r%r;r!rg#B ;rrr2rzr7r3rr4r6rrrrr)r<rWrrrr2rr?r)!rrrrrErrrrr matrix_rankrrrrr r rrGr rQrrr*rrrErrrrrr)(rrrHr^rrrr4r3rrWrrrrr?rr&r-r$r%r;rrrbiginddeltardirec1fx2rZbndrD_rttempr<rr=s( rArXrXU sx?+ F F A # AA6>d(T  RVV $hGffG  bff XFVVF;4vNLFD }AU#U+ 99  '5;;q> 9 MM-)a 9~ $. [$*99fii[ 66+" #rvvb;.>'? MML)a 9 7D B D = BFE1X q"4T1f9=ALAL:> #@a $J%'$JEF  AIDq!"01$"? (3FG"&&*rvvd|34u Et&)#+h!m$'1.F#y MA!   s7'|dkDr9tj|| dd}tj| j| } dddt}t'|j)d }tj*|t,}tj*|t.}t|dz ddD]} || }||| z|| <||z}t|D]} | | t ||| <|t |}|dk(r | d} |d}t1|rt3|j4}t7}d |vrd|d <d |vr||d <n d |vrd |i|d <|||fd |i|}t |t8r%|j:}|j<}|j>}n|d}|d}|ddk(}|s|rtAjBdtDd|r||| |fS|S#1swYxYw)aMinimize a function over a given range by brute force. Uses the "brute force" method, i.e., computes the function's value at each point of a multidimensional grid of points, to find the global minimum of the function. The function is evaluated everywhere in the range with the datatype of the first call to the function, as enforced by the ``vectorize`` NumPy function. The value and type of the function evaluation returned when ``full_output=True`` are affected in addition by the ``finish`` argument (see Notes). The brute force approach is inefficient because the number of grid points increases exponentially - the number of grid points to evaluate is ``Ns ** len(x)``. Consequently, even with coarse grid spacing, even moderately sized problems can take a long time to run, and/or run into memory limitations. Parameters ---------- func : callable The objective function to be minimized. Must be in the form ``f(x, *args)``, where ``x`` is the argument in the form of a 1-D array and ``args`` is a tuple of any additional fixed parameters needed to completely specify the function. ranges : tuple Each component of the `ranges` tuple must be either a "slice object" or a range tuple of the form ``(low, high)``. The program uses these to create the grid of points on which the objective function will be computed. See `Note 2` for more detail. args : tuple, optional Any additional fixed parameters needed to completely specify the function. Ns : int, optional Number of grid points along the axes, if not otherwise specified. See `Note2`. full_output : bool, optional If True, return the evaluation grid and the objective function's values on it. finish : callable, optional An optimization function that is called with the result of brute force minimization as initial guess. `finish` should take `func` and the initial guess as positional arguments, and take `args` as keyword arguments. It may additionally take `full_output` and/or `disp` as keyword arguments. Use None if no "polishing" function is to be used. See Notes for more details. disp : bool, optional Set to True to print convergence messages from the `finish` callable. workers : int or map-like callable, optional If `workers` is an int the grid is subdivided into `workers` sections and evaluated in parallel (uses `multiprocessing.Pool `). Supply `-1` to use all cores available to the Process. Alternatively supply a map-like callable, such as `multiprocessing.Pool.map` for evaluating the grid in parallel. This evaluation is carried out as ``workers(func, iterable)``. Requires that `func` be pickleable. .. versionadded:: 1.3.0 Returns ------- x0 : ndarray A 1-D array containing the coordinates of a point at which the objective function had its minimum value. (See `Note 1` for which point is returned.) fval : float Function value at the point `x0`. (Returned when `full_output` is True.) grid : tuple Representation of the evaluation grid. It has the same length as `x0`. (Returned when `full_output` is True.) Jout : ndarray Function values at each point of the evaluation grid, i.e., ``Jout = func(*grid)``. (Returned when `full_output` is True.) See Also -------- basinhopping, differential_evolution Notes ----- *Note 1*: The program finds the gridpoint at which the lowest value of the objective function occurs. If `finish` is None, that is the point returned. When the global minimum occurs within (or not very far outside) the grid's boundaries, and the grid is fine enough, that point will be in the neighborhood of the global minimum. However, users often employ some other optimization program to "polish" the gridpoint values, i.e., to seek a more precise (local) minimum near `brute's` best gridpoint. The `brute` function's `finish` option provides a convenient way to do that. Any polishing program used must take `brute's` output as its initial guess as a positional argument, and take `brute's` input values for `args` as keyword arguments, otherwise an error will be raised. It may additionally take `full_output` and/or `disp` as keyword arguments. `brute` assumes that the `finish` function returns either an `OptimizeResult` object or a tuple in the form: ``(xmin, Jmin, ... , statuscode)``, where ``xmin`` is the minimizing value of the argument, ``Jmin`` is the minimum value of the objective function, "..." may be some other returned values (which are not used by `brute`), and ``statuscode`` is the status code of the `finish` program. Note that when `finish` is not None, the values returned are those of the `finish` program, *not* the gridpoint ones. Consequently, while `brute` confines its search to the input grid points, the `finish` program's results usually will not coincide with any gridpoint, and may fall outside the grid's boundary. Thus, if a minimum only needs to be found over the provided grid points, make sure to pass in ``finish=None``. *Note 2*: The grid of points is a `numpy.mgrid` object. For `brute` the `ranges` and `Ns` inputs have the following effect. Each component of the `ranges` tuple can be either a slice object or a two-tuple giving a range of values, such as (0, 5). If the component is a slice object, `brute` uses it directly. If the component is a two-tuple range, `brute` internally converts it to a slice object that interpolates `Ns` points from its low-value to its high-value, inclusive. Examples -------- We illustrate the use of `brute` to seek the global minimum of a function of two variables that is given as the sum of a positive-definite quadratic and two deep "Gaussian-shaped" craters. Specifically, define the objective function `f` as the sum of three other functions, ``f = f1 + f2 + f3``. We suppose each of these has a signature ``(z, *params)``, where ``z = (x, y)``, and ``params`` and the functions are as defined below. >>> import numpy as np >>> params = (2, 3, 7, 8, 9, 10, 44, -1, 2, 26, 1, -2, 0.5) >>> def f1(z, *params): ... x, y = z ... a, b, c, d, e, f, g, h, i, j, k, l, scale = params ... return (a * x**2 + b * x * y + c * y**2 + d*x + e*y + f) >>> def f2(z, *params): ... x, y = z ... a, b, c, d, e, f, g, h, i, j, k, l, scale = params ... return (-g*np.exp(-((x-h)**2 + (y-i)**2) / scale)) >>> def f3(z, *params): ... x, y = z ... a, b, c, d, e, f, g, h, i, j, k, l, scale = params ... return (-j*np.exp(-((x-k)**2 + (y-l)**2) / scale)) >>> def f(z, *params): ... return f1(z, *params) + f2(z, *params) + f3(z, *params) Thus, the objective function may have local minima near the minimum of each of the three functions of which it is composed. To use `fmin` to polish its gridpoint result, we may then continue as follows: >>> rranges = (slice(-4, 4, 0.25), slice(-4, 4, 0.25)) >>> from scipy import optimize >>> resbrute = optimize.brute(f, rranges, args=params, full_output=True, ... finish=optimize.fmin) >>> resbrute[0] # global minimum array([-1.05665192, 1.80834843]) >>> resbrute[1] # function value at global minimum -3.4085818767 Note that if `finish` had been set to None, we would have gotten the gridpoint [-1.0 1.75] where the rounded function value is -2.892. (z5Brute Force not possible with more than 40 variables.rr!rN)poolrrrrrrHzhEither final optimization did not succeed or `finish` does not return `statuscode` as its last argument.rr{)#rrvrr rslicerNcomplexrEmgridrreshapeprodTiterable_Brute_Wrapperr'r rrrr rrr_getfullargspecrHdictrr?r<r2rrr)rrangesrHNsrfinishrrr&lranger(grid inpt_shape wrapped_funcmapperJoutNshapeindxNindxrthisNJmin finish_args finish_kwargsr]r2s rArrYsVZ F A2v./ / &\F 1X*&)U+6!9~!!&), ~=q vay)F1I * Q 88F DJ Azz$A 120G HIKK ;;t w!$-L  !2VxxVL$789 F7D::d#D!e::dJqrN3D::dffj1D24[F $**,R (D HHQ E 88Au D 1q5"b !q &)#au}1X(q'%,'Q( e D QAwAw%f-22  K '+,M- ( [ $(M& ! + %)/~M) $T4 *55D77DkkGq6Dq6D"glG S,<T4%% w22s !BL55L?ceZdZdZdZdZy)rqzU Object to wrap user cost function for optimize.brute, allowing picklability c4||_|g|_y||_yr;)rrH)r@rrHs rArBz_Brute_Wrapper.__init__cs,B D rCc~|jtj|jg|jSr;)rrErrrH)r@r?s rArMz_Brute_Wrapper.__call__gs-tvvbjjm++-: ::rCN)rPrQrRrSrBrMrTrCrArqrq^s1;rCrqc >ddl}ddddddd }|gd }|jtd d |jgd|jtdd |jgd|jtdd |jgd|jtdd dj |}n?|j }||vrt d||fg}||D]J\}}|jd|zddt|zzdzg|jt||d Ldj |}n|j }t||}||vrt d|||}|jd} dj | dd} t| ttj| | d} | j} | |j| j!}nd}|r t#|y|S)a Show documentation for additional options of optimization solvers. These are method-specific options that can be supplied through the ``options`` dict. Parameters ---------- solver : str Type of optimization solver. One of 'minimize', 'minimize_scalar', 'root', 'root_scalar', 'linprog', or 'quadratic_assignment'. method : str, optional If not given, shows all methods of the specified solver. Otherwise, show only the options for the specified method. Valid values corresponds to methods' names of respective solver (e.g., 'BFGS' for 'minimize'). disp : bool, optional Whether to print the result rather than returning it. Returns ------- text Either None (for disp=True) or the text string (disp=False) Notes ----- The solver-specific methods are: `scipy.optimize.minimize` - :ref:`Nelder-Mead ` - :ref:`Powell ` - :ref:`CG ` - :ref:`BFGS ` - :ref:`Newton-CG ` - :ref:`L-BFGS-B ` - :ref:`TNC ` - :ref:`COBYLA ` - :ref:`COBYQA ` - :ref:`SLSQP ` - :ref:`dogleg ` - :ref:`trust-ncg ` `scipy.optimize.root` - :ref:`hybr ` - :ref:`lm ` - :ref:`broyden1 ` - :ref:`broyden2 ` - :ref:`anderson ` - :ref:`linearmixing ` - :ref:`diagbroyden ` - :ref:`excitingmixing ` - :ref:`krylov ` - :ref:`df-sane ` `scipy.optimize.minimize_scalar` - :ref:`brent ` - :ref:`golden ` - :ref:`bounded ` `scipy.optimize.root_scalar` - :ref:`bisect ` - :ref:`brentq ` - :ref:`brenth ` - :ref:`ridder ` - :ref:`toms748 ` - :ref:`newton ` - :ref:`secant ` - :ref:`halley ` `scipy.optimize.linprog` - :ref:`simplex ` - :ref:`interior-point ` - :ref:`revised simplex ` - :ref:`highs ` - :ref:`highs-ds ` - :ref:`highs-ipm ` `scipy.optimize.quadratic_assignment` - :ref:`faq ` - :ref:`2opt ` Examples -------- We can print documentations of a solver in stdout: >>> from scipy.optimize import show_options >>> show_options(solver="minimize") ... Specifying a method is possible: >>> show_options(solver="minimize", method="Nelder-Mead") ... We can also get the documentations as a string: >>> show_options(solver="minimize", method="Nelder-Mead", disp=False) Minimization of scalar function of one or more variables using the ... rN))bfgsz'scipy.optimize._optimize._minimize_bfgs)cgz%scipy.optimize._optimize._minimize_cg)rXz*scipy.optimize._cobyla_py._minimize_cobyla)rYz*scipy.optimize._cobyqa_py._minimize_cobyqa)doglegz3scipy.optimize._trustregion_dogleg._minimize_dogleg)zl-bfgs-bz*scipy.optimize._lbfgsb_py._minimize_lbfgsb)z nelder-meadz-scipy.optimize._optimize._minimize_neldermead)z newton-cgz+scipy.optimize._optimize._minimize_newtoncg)powellz)scipy.optimize._optimize._minimize_powell)rWz(scipy.optimize._slsqp_py._minimize_slsqp)rVz!scipy.optimize._tnc._minimize_tnc)z trust-ncgz3scipy.optimize._trustregion_ncg._minimize_trust_ncg)r`z?scipy.optimize._trustregion_constr._minimize_trustregion_constr)z trust-exactz=scipy.optimize._trustregion_exact._minimize_trustregion_exact)z trust-krylovz9scipy.optimize._trustregion_krylov._minimize_trust_krylov) )hybrz%scipy.optimize._minpack_py._root_hybr)lmz"scipy.optimize._root._root_leastsq)broyden1z'scipy.optimize._root._root_broyden1_doc)broyden2z'scipy.optimize._root._root_broyden2_doc)andersonz'scipy.optimize._root._root_anderson_doc) diagbroydenz*scipy.optimize._root._root_diagbroyden_doc)excitingmixingz-scipy.optimize._root._root_excitingmixing_doc) linearmixingz+scipy.optimize._root._root_linearmixing_doc)krylovz%scipy.optimize._root._root_krylov_doc)zdf-sanez&scipy.optimize._spectral._root_df_sane))bisectz3scipy.optimize._root_scalar._root_scalar_bisect_doc)brentqz3scipy.optimize._root_scalar._root_scalar_brentq_doc)brenthz3scipy.optimize._root_scalar._root_scalar_brenth_doc)ridderz3scipy.optimize._root_scalar._root_scalar_ridder_doc)toms748z4scipy.optimize._root_scalar._root_scalar_toms748_doc)secantz3scipy.optimize._root_scalar._root_scalar_secant_doc)newtonz3scipy.optimize._root_scalar._root_scalar_newton_doc)halleyz3scipy.optimize._root_scalar._root_scalar_halley_doc))simplexz,scipy.optimize._linprog._linprog_simplex_doc)zinterior-pointz'scipy.optimize._linprog._linprog_ip_doc)zrevised simplexz'scipy.optimize._linprog._linprog_rs_doc)z highs-ipmz.scipy.optimize._linprog._linprog_highs_ipm_doc)zhighs-dsz-scipy.optimize._linprog._linprog_highs_ds_doc)highsz*scipy.optimize._linprog._linprog_highs_doc))faqz-scipy.optimize._qap._quadratic_assignment_faq)2optz.scipy.optimize._qap._quadratic_assignment_2opt))rz/scipy.optimize._optimize._minimize_scalar_brent)boundedz1scipy.optimize._optimize._minimize_scalar_bounded)r z0scipy.optimize._optimize._minimize_scalar_golden)minimizeroot root_scalarlinprogquadratic_assignmentminimize_scalar)z ======== z minimize z ======== rF)r)z =============== zminimize_scalar z=============== r)z ==== zroot z==== r)z ======= zlinprog z======= rrzUnknown solver z rf=zUnknown method .r)textwraprrextendr}lowerrvrrssplit __import__getattrsysmodulesrSdedentstripr) r5rlrr doc_routinestextnameramethodspartsmod_nameobjdocs rArrlsV ,     !  s>L@~? L%89 * + L!2?@   Le45 " # L78wwt}  %vj9: : >D'/ Da Vd]D3s4y=,@6,IJK LEBC D774=D\\^F</0GW$ ?6*!=>>6?DJJsOExxcr +H x #++h/r;C++Cs+113  d  rCr;)r)NrTNNNNN) rTrrNNrr!rNN) rTNNNFFNrrFN)rTrrrrr!)rTrrrr)rTNrrr)NrTrrr)rrrTg[@rZ)gMbP?NNN) rTNNrrNNFNF)NNT)g__all__ __docformat__rrrrgnumpyrrrrrrrE scipy.linalgrrrscipy.sparse.linalgr _linesearchr"r#rr$_numdiffr%scipy._lib._utilr&rrr'r(r)r*r+(scipy.optimize._differentiable_functionsr,r-scipy._lib._array_apir.r/r0 scipy._libr1rrr9rnr UserWarningrrxrrrrrv_epsilonrrrrr r r rrr'rrrrrrr\r^rqrrr{rrrrrrrrrrr r r r(r9rErQrrXrrrqrrTrCrArs(  &  ::;;.--(FMMONN-F/*33%. /60* [* Z k  LW. ## $;AE@D04nb  . . b STU%V%P STU'4V'4T STU)V)X$0  $6HLIMzz6:9>;?HLK\"*N/bF+.6TZ,Zz | *Z!r266!!Q4CM`"$266x%dt!4 upJRdd!dstn "tdXtDS$_D%)2tt1tup&(TD $4Xt!ed!D8;"#[|13;<SlrrjM`