a jנ @sdZddlZddlZddlZddlZddlmZmZddlZ ddl m Z ddl m Z ddlmZddlmZdd lmZdd lmZmZdd lmZdd lmZmZmZdd lmZmZm Z m!Z!m"Z"ddl#m$Z$m%Z%ddl&m'Z'm(Z(m)Z)m*Z*ddl+m,Z,m-Z-m.Z.ddZ/ddZ0dddddde 1e j2j3dddZ4ddZ5edgd gd gd!dd"dddddde 1e j2j3dd#d$d%Z6Gd&d'd'e,Z7Gd(d)d)e7Z8ddddddde 1e j2j3fd*d+Z9Gd,d-d-e7Z:dS).zUGraphicalLasso: sparse inverse covariance estimation with an l1-penalized estimator. N)IntegralReal)linalg) _fit_context)ConvergenceWarning)_cd_fast)lars_path_gram)check_cvcross_val_score)Bunch)Interval StrOptionsvalidate_params)MetadataRouter MethodMapping_raise_for_params_routing_enabledprocess_routing)Paralleldelayed)_is_arraylike_not_scalarcheck_random_state check_scalar validate_data)EmpiricalCovarianceempirical_covariancelog_likelihoodcCsZ|jd}dt|||tdtj}||t|tt|7}|S)zEvaluation of the graphical-lasso objective function the objective function is made of a shifted scaled version of the normalized log-likelihood (i.e. its empirical mean over the samples) and a penalisation term to promote sparsity rr)shapernplogpiabssumdiag)Zmle precision_alphapcostr+EE:\soccer\ml_env\Lib\site-packages\sklearn/covariance/_graph_lasso.py _objective-s "*r-cCsJt||}||jd8}||t|tt|7}|S)zExpression of the dual gap convergence criterion The specific definition is given in Duchi "Projected Subgradient Methods for Learning Sparse Gaussians". r)r!r%r r$r&)emp_covr'r(gapr+r+r, _dual_gap:s*r0cd-C6?dF)cov_initmodetolenet_tolmax_iterverboseepscCs|j\} } |dkrdt|} dt|| } | | tdtj7} t|| | } || | | fdfS|durv|}n|}|d9}|j dd| d}||j dd| d<t |} t | }d}t }|dkrt ddd }n t dd }ztj} tj|ddddfd d }t|D]f}t| D]}|dkr||d}||||k||<|dd|f||k|dd|f<n|ddddf|dd<||||kf}tjfi||dkr| ||k|f| ||fd | }t||d|||||tdd \}} } } n(t|||j|| dd|ddd\} } }Wdn1sR0Yd|||ft|||k|f|| ||f<| ||f || ||k|f<| ||f || |||kf<t||}|||||kf<||||k|f<q*t| stdt|| |} t|| |} |r@td|| | f|| | ft| |krdqt| s|dkrtdqtd|| ft Wn<ty}z"|j!ddf|_!|WYd}~n d}~00|| ||dfS)Nrrrgffffff?rr1raiseignore)Zoverinvalid)r=C)orderiFTlars)ZXyZGramZ n_samplesZ alpha_minZ copy_Gramr:methodZ return_pathg?z1The system is too ill-conditioned for this solverz<[graphical_lasso] Iteration % 3i, cost % 3.2e, dual gap %.3ezANon SPD result: the system is too ill-conditioned for this solverzDgraphical_lasso: did not converge after %i iteration: dual gap: %.3ez3. The system is too ill-conditioned for this solver)"r rinvrr!r"r#r%copyflatZpinvhZarangelistdictinfrangeZerrstatecd_fastZenet_coordinate_descent_gramrr sizedotisfiniteFloatingPointErrorr0r-printappendr$warningswarnrargs)r.r(r4r5r6r7r8r9r:_Z n_featuresr'r*Zd_gap covariance_ZdiagonalindicesiZcostserrorsZsub_covarianceidxZdirowZcoefser+r+r,_graphical_lassoGs       &   ,      r[cCs4t|}d|jdd|jdd<tt|S)aFind the maximum alpha for which there are some non-zeros off-diagonal. Parameters ---------- emp_cov : ndarray of shape (n_features, n_features) The sample covariance matrix. Notes ----- This results from the bound for the all the Lasso that are solved in GraphicalLasso: each time, the row of cov corresponds to Xy. As the bound for alpha is given by `max(abs(Xy))`, the result follows. rNr)r!rCrDr maxr$)r.Ar+r+r, alpha_maxs r^ array-likeboolean)r. return_costs return_n_iterZprefer_skip_nested_validation)r5r6r7r8r9rar:rbc CsTt||d|||||dd |} | j| jg} |r<| | j| rL| | jt| S)a<L1-penalized covariance estimator. Read more in the :ref:`User Guide `. .. versionchanged:: v0.20 graph_lasso has been renamed to graphical_lasso Parameters ---------- emp_cov : array-like of shape (n_features, n_features) Empirical covariance from which to compute the covariance estimate. alpha : float The regularization parameter: the higher alpha, the more regularization, the sparser the inverse covariance. Range is (0, inf]. mode : {'cd', 'lars'}, default='cd' The Lasso solver to use: coordinate descent or LARS. Use LARS for very sparse underlying graphs, where p > n. Elsewhere prefer cd which is more numerically stable. tol : float, default=1e-4 The tolerance to declare convergence: if the dual gap goes below this value, iterations are stopped. Range is (0, inf]. enet_tol : float, default=1e-4 The tolerance for the elastic net solver used to calculate the descent direction. This parameter controls the accuracy of the search direction for a given column update, not of the overall parameter estimate. Only used for mode='cd'. Range is (0, inf]. max_iter : int, default=100 The maximum number of iterations. verbose : bool, default=False If verbose is True, the objective function and dual gap are printed at each iteration. return_costs : bool, default=False If return_costs is True, the objective function and dual gap at each iteration are returned. eps : float, default=eps The machine-precision regularization in the computation of the Cholesky diagonal factors. Increase this for very ill-conditioned systems. Default is `np.finfo(np.float64).eps`. return_n_iter : bool, default=False Whether or not to return the number of iterations. Returns ------- covariance : ndarray of shape (n_features, n_features) The estimated covariance matrix. precision : ndarray of shape (n_features, n_features) The estimated (sparse) precision matrix. costs : list of (objective, dual_gap) pairs The list of values of the objective function and the dual gap at each iteration. Returned only if return_costs is True. n_iter : int Number of iterations. Returned only if `return_n_iter` is set to True. See Also -------- GraphicalLasso : Sparse inverse covariance estimation with an l1-penalized estimator. GraphicalLassoCV : Sparse inverse covariance with cross-validated choice of the l1 penalty. Notes ----- The algorithm employed to solve this problem is the GLasso algorithm, from the Friedman 2008 Biostatistics paper. It is the same algorithm as in the R `glasso` package. One possible difference with the `glasso` R package is that the diagonal coefficients are not penalized. Examples -------- >>> import numpy as np >>> from sklearn.datasets import make_sparse_spd_matrix >>> from sklearn.covariance import empirical_covariance, graphical_lasso >>> true_cov = make_sparse_spd_matrix(n_dim=3,random_state=42) >>> rng = np.random.RandomState(42) >>> X = rng.multivariate_normal(mean=np.zeros(3), cov=true_cov, size=3) >>> emp_cov = empirical_covariance(X, assume_centered=True) >>> emp_cov, _ = graphical_lasso(emp_cov, alpha=0.05) >>> emp_cov array([[ 1.68..., 0.21..., -0.20...], [ 0.21..., 0.22..., -0.08...], [-0.20..., -0.08..., 0.23...]]) precomputedT) r(r5 covariancer6r7r8r9r:assume_centered)GraphicalLassofitrTr'rOcosts_n_iter_tuple) r.r(r5r6r7r8r9rar:rbmodeloutputr+r+r,graphical_lassos&v    rnc seZdZUiejeeddddgeeddddgeeddddgeddhgdgeeddd dgd Ze e d <e d d d ddde e jjdffdd ZZS)BaseGraphicalLassorNrightclosedleftr1r@r9both)r6r7r8r5r9r:_parameter_constraintsZstore_precisionr2r3Fcs6tj|d||_||_||_||_||_||_dS)Nrf)super__init__r6r7r8r5r9r:)selfr6r7r8r5r9r:rf __class__r+r,rxus zBaseGraphicalLasso.__init__)__name__ __module__ __qualname__rrur rrrrF__annotations__popr!finfofloat64r:rx __classcell__r+r+rzr,rois&    roc seZdZUdZiejeeddddgedhdgdZe e d<dd dd d d d e e j jd dfdd ZedddddZZS)rgagSparse inverse covariance estimation with an l1-penalized estimator. For a usage example see :ref:`sphx_glr_auto_examples_applications_plot_stock_market.py`. Read more in the :ref:`User Guide `. .. versionchanged:: v0.20 GraphLasso has been renamed to GraphicalLasso Parameters ---------- alpha : float, default=0.01 The regularization parameter: the higher alpha, the more regularization, the sparser the inverse covariance. Range is (0, inf]. mode : {'cd', 'lars'}, default='cd' The Lasso solver to use: coordinate descent or LARS. Use LARS for very sparse underlying graphs, where p > n. Elsewhere prefer cd which is more numerically stable. covariance : "precomputed", default=None If covariance is "precomputed", the input data in `fit` is assumed to be the covariance matrix. If `None`, the empirical covariance is estimated from the data `X`. .. versionadded:: 1.3 tol : float, default=1e-4 The tolerance to declare convergence: if the dual gap goes below this value, iterations are stopped. Range is (0, inf]. enet_tol : float, default=1e-4 The tolerance for the elastic net solver used to calculate the descent direction. This parameter controls the accuracy of the search direction for a given column update, not of the overall parameter estimate. Only used for mode='cd'. Range is (0, inf]. max_iter : int, default=100 The maximum number of iterations. verbose : bool, default=False If verbose is True, the objective function and dual gap are plotted at each iteration. eps : float, default=eps The machine-precision regularization in the computation of the Cholesky diagonal factors. Increase this for very ill-conditioned systems. Default is `np.finfo(np.float64).eps`. .. versionadded:: 1.3 assume_centered : bool, default=False If True, data are not centered before computation. Useful when working with data whose mean is almost, but not exactly zero. If False, data are centered before computation. Attributes ---------- location_ : ndarray of shape (n_features,) Estimated location, i.e. the estimated mean. covariance_ : ndarray of shape (n_features, n_features) Estimated covariance matrix precision_ : ndarray of shape (n_features, n_features) Estimated pseudo inverse matrix. n_iter_ : int Number of iterations run. costs_ : list of (objective, dual_gap) pairs The list of values of the objective function and the dual gap at each iteration. Returned only if return_costs is True. .. versionadded:: 1.3 n_features_in_ : int Number of features seen during :term:`fit`. .. versionadded:: 0.24 feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- graphical_lasso : L1-penalized covariance estimator. GraphicalLassoCV : Sparse inverse covariance with cross-validated choice of the l1 penalty. Examples -------- >>> import numpy as np >>> from sklearn.covariance import GraphicalLasso >>> true_cov = np.array([[0.8, 0.0, 0.2, 0.0], ... [0.0, 0.4, 0.0, 0.0], ... [0.2, 0.0, 0.3, 0.1], ... [0.0, 0.0, 0.1, 0.7]]) >>> np.random.seed(0) >>> X = np.random.multivariate_normal(mean=[0, 0, 0, 0], ... cov=true_cov, ... size=200) >>> cov = GraphicalLasso().fit(X) >>> np.around(cov.covariance_, decimals=3) array([[0.816, 0.049, 0.218, 0.019], [0.049, 0.364, 0.017, 0.034], [0.218, 0.017, 0.322, 0.093], [0.019, 0.034, 0.093, 0.69 ]]) >>> np.around(cov.location_, decimals=3) array([0.073, 0.04 , 0.038, 0.143]) rNrtrqrd)r(reru{Gz?r1r2r3F)r5rer6r7r8r9r:rfc s*tj||||||| d||_||_dSN)r6r7r8r5r9r:rf)rwrxr(re) ryr(r5rer6r7r8r9r:rfrzr+r,rxs  zGraphicalLasso.__init__Trcc Cst||ddd}|jdkr6|}t|jd|_n4t||jd}|jr^t|jd|_n | d|_t ||j d|j |j |j|j|j|jd \|_|_|_|_|S) aFit the GraphicalLasso model to X. Parameters ---------- X : array-like of shape (n_samples, n_features) Data from which to compute the covariance estimate. y : Ignored Not used, present for API consistency by convention. Returns ------- self : object Returns the instance itself. r)ensure_min_featuresZensure_min_samplesrdrrvrNr(r4r5r6r7r8r9r:)rrerCr!zerosr location_rrfmeanr[r(r5r6r7r8r9r:rTr'rirj)ryXyr.r+r+r,rhs(   zGraphicalLasso.fit)r)N)r|r}r~__doc__rorur rrrFrr!rrr:rxrrhrr+r+rzr,rgs( v  rgc  Cs`td|d} t|} |dur(| } n|} t} t}t}|durNt|}|D]}zLt| || ||||| | d \} }}}| | |||durt||}Wn2tytj }| tj |tj Yn0|durt |stj }|||dkrt j dqR|dkrR|dur6td||fqRtd|qR|durX| ||fS| |fS)a l1-penalized covariance estimator along a path of decreasing alphas Read more in the :ref:`User Guide `. Parameters ---------- X : ndarray of shape (n_samples, n_features) Data from which to compute the covariance estimate. alphas : array-like of shape (n_alphas,) The list of regularization parameters, decreasing order. cov_init : array of shape (n_features, n_features), default=None The initial guess for the covariance. X_test : array of shape (n_test_samples, n_features), default=None Optional test matrix to measure generalisation error. mode : {'cd', 'lars'}, default='cd' The Lasso solver to use: coordinate descent or LARS. Use LARS for very sparse underlying graphs, where p > n. Elsewhere prefer cd which is more numerically stable. tol : float, default=1e-4 The tolerance to declare convergence: if the dual gap goes below this value, iterations are stopped. The tolerance must be a positive number. enet_tol : float, default=1e-4 The tolerance for the elastic net solver used to calculate the descent direction. This parameter controls the accuracy of the search direction for a given column update, not of the overall parameter estimate. Only used for mode='cd'. The tolerance must be a positive number. max_iter : int, default=100 The maximum number of iterations. This parameter should be a strictly positive integer. verbose : int or bool, default=False The higher the verbosity flag, the more information is printed during the fitting. eps : float, default=eps The machine-precision regularization in the computation of the Cholesky diagonal factors. Increase this for very ill-conditioned systems. Default is `np.finfo(np.float64).eps`. .. versionadded:: 1.3 Returns ------- covariances_ : list of shape (n_alphas,) of ndarray of shape (n_features, n_features) The estimated covariance matrices. precisions_ : list of shape (n_alphas,) of ndarray of shape (n_features, n_features) The estimated (sparse) precision matrices. scores_ : list of shape (n_alphas,), dtype=float The generalisation error (log-likelihood) on the test data. Returned only if test data is passed. rrNr.z/[graphical_lasso_path] alpha: %.2e, score: %.2ez"[graphical_lasso_path] alpha: %.2e)r\rrCrEr[rOrrMr!rGnanrLsysstderrwriterN)ralphasr4X_testr5r6r7r8r9r: inner_verboser.rTZ covariances_Z precisions_Zscores_Z test_emp_covr(r'rS this_scorer+r+r,graphical_lasso_pathKs`K           rc seZdZUdZiejeedddddgeeddddgdgedgd Zee d <d d dd d d ddde e j j dd fdd ZedddddZddZZS)GraphicalLassoCVa?Sparse inverse covariance w/ cross-validated choice of the l1 penalty. See glossary entry for :term:`cross-validation estimator`. Read more in the :ref:`User Guide `. .. versionchanged:: v0.20 GraphLassoCV has been renamed to GraphicalLassoCV Parameters ---------- alphas : int or array-like of shape (n_alphas,), dtype=float, default=4 If an integer is given, it fixes the number of points on the grids of alpha to be used. If a list is given, it gives the grid to be used. See the notes in the class docstring for more details. Range is [1, inf) for an integer. Range is (0, inf] for an array-like of floats. n_refinements : int, default=4 The number of times the grid is refined. Not used if explicit values of alphas are passed. Range is [1, inf). cv : int, cross-validation generator or iterable, default=None Determines the cross-validation splitting strategy. Possible inputs for cv are: - None, to use the default 5-fold cross-validation, - integer, to specify the number of folds. - :term:`CV splitter`, - An iterable yielding (train, test) splits as arrays of indices. For integer/None inputs :class:`~sklearn.model_selection.KFold` is used. Refer :ref:`User Guide ` for the various cross-validation strategies that can be used here. .. versionchanged:: 0.20 ``cv`` default value if None changed from 3-fold to 5-fold. tol : float, default=1e-4 The tolerance to declare convergence: if the dual gap goes below this value, iterations are stopped. Range is (0, inf]. enet_tol : float, default=1e-4 The tolerance for the elastic net solver used to calculate the descent direction. This parameter controls the accuracy of the search direction for a given column update, not of the overall parameter estimate. Only used for mode='cd'. Range is (0, inf]. max_iter : int, default=100 Maximum number of iterations. mode : {'cd', 'lars'}, default='cd' The Lasso solver to use: coordinate descent or LARS. Use LARS for very sparse underlying graphs, where number of features is greater than number of samples. Elsewhere prefer cd which is more numerically stable. n_jobs : int, default=None Number of jobs to run in parallel. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. .. versionchanged:: v0.20 `n_jobs` default changed from 1 to None verbose : bool, default=False If verbose is True, the objective function and duality gap are printed at each iteration. eps : float, default=eps The machine-precision regularization in the computation of the Cholesky diagonal factors. Increase this for very ill-conditioned systems. Default is `np.finfo(np.float64).eps`. .. versionadded:: 1.3 assume_centered : bool, default=False If True, data are not centered before computation. Useful when working with data whose mean is almost, but not exactly zero. If False, data are centered before computation. Attributes ---------- location_ : ndarray of shape (n_features,) Estimated location, i.e. the estimated mean. covariance_ : ndarray of shape (n_features, n_features) Estimated covariance matrix. precision_ : ndarray of shape (n_features, n_features) Estimated precision matrix (inverse covariance). costs_ : list of (objective, dual_gap) pairs The list of values of the objective function and the dual gap at each iteration. Returned only if return_costs is True. .. versionadded:: 1.3 alpha_ : float Penalization parameter selected. cv_results_ : dict of ndarrays A dict with keys: alphas : ndarray of shape (n_alphas,) All penalization parameters explored. split(k)_test_score : ndarray of shape (n_alphas,) Log-likelihood score on left-out data across (k)th fold. .. versionadded:: 1.0 mean_test_score : ndarray of shape (n_alphas,) Mean of scores over the folds. .. versionadded:: 1.0 std_test_score : ndarray of shape (n_alphas,) Standard deviation of scores over the folds. .. versionadded:: 1.0 n_iter_ : int Number of iterations run for the optimal alpha. n_features_in_ : int Number of features seen during :term:`fit`. .. versionadded:: 0.24 feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- graphical_lasso : L1-penalized covariance estimator. GraphicalLasso : Sparse inverse covariance estimation with an l1-penalized estimator. Notes ----- The search for the optimal penalization parameter (`alpha`) is done on an iteratively refined grid: first the cross-validated scores on a grid are computed, then a new refined grid is centered around the maximum, and so on. One of the challenges which is faced here is that the solvers can fail to converge to a well-conditioned estimate. The corresponding values of `alpha` then come out as missing values, but the optimum may be close to these missing values. In `fit`, once the best parameter `alpha` is found through cross-validation, the model is fit again using the entire training set. Examples -------- >>> import numpy as np >>> from sklearn.covariance import GraphicalLassoCV >>> true_cov = np.array([[0.8, 0.0, 0.2, 0.0], ... [0.0, 0.4, 0.0, 0.0], ... [0.2, 0.0, 0.3, 0.1], ... [0.0, 0.0, 0.1, 0.7]]) >>> np.random.seed(0) >>> X = np.random.multivariate_normal(mean=[0, 0, 0, 0], ... cov=true_cov, ... size=200) >>> cov = GraphicalLassoCV().fit(X) >>> np.around(cov.covariance_, decimals=3) array([[0.816, 0.051, 0.22 , 0.017], [0.051, 0.364, 0.018, 0.036], [0.22 , 0.018, 0.322, 0.094], [0.017, 0.036, 0.094, 0.69 ]]) >>> np.around(cov.location_, decimals=3) array([0.073, 0.04 , 0.038, 0.143]) rNrsrqr_rZ cv_object)r n_refinementscvn_jobsrur2r3r1F) rrrr6r7r8r5rr9r:rfc s6tj||||| | | d||_||_||_||_dSr)rwrxrrrr) ryrrrr6r7r8r5rr9r:rfrzr+r,rxs zGraphicalLassoCV.__init__Trcc snt|dtddjr4tjd_n d_tjd}t j |dd}t }j }t djdt|rj D]}t|d tdtjd d qj d} n:j} t|} d | } tt| t| |d d dtrtdfi|} nttidd} t} t| D]>}tXtdtt j!jdfdd|j"|fi| j#j"D}Wd n1s0Yt$|\}}}t$|}t$|}|%t$||t&|t'(ddd}tj }d}t)|D]Z\}\}}}t|}|dt*tj+j,kr,tj-}t.|r<|}||kr|}|}q|dkrv|dd} |dd} n||kr|t/|dks||d} ||dd} nP|t/|dkr||d} d ||d} n ||dd} ||dd} t|s4tt| t| |dddjr&| dkr&t0d|d| t| fq&t t$|}t |d}t |d1d|1t2t3|j!|dt4|}dt4i_5t|jdD]$}|d d |fj5d|d<qtj|ddj5d<tj6|ddj5d<|}|_7t8||j9j:j;j<j,d \_=_>_?_@S)!aXFit the GraphicalLasso covariance model to X. Parameters ---------- X : array-like of shape (n_samples, n_features) Data from which to compute the covariance estimate. y : Ignored Not used, present for API consistency by convention. **params : dict, default=None Parameters to be passed to the CV splitter and the cross_val_score function. .. versionadded:: 1.5 Only available if `enable_metadata_routing=True`, which can be set by using ``sklearn.set_config(enable_metadata_routing=True)``. See :ref:`Metadata Routing User Guide ` for more details. Returns ------- self : object Returns the instance itself. rhr)rrrrvF) classifierr(rp)Zmin_valZmax_valZinclude_boundariesrN)split)splitterr<)rr9c 3sJ|]B\}}tt||jjjtdjjd VqdS)皙?)rrr5r6r7r8r9r:N)rrr5r6r7intr8r:).0traintestrrrryr+r, s  z'GraphicalLassoCV.fit..T)keyreverserz8[GraphicalLassoCV] Done refinement % 2i out of %i: % 3is)rrr9paramsrrZ _test_score)ZaxisZmean_test_scoreZstd_test_score)r(r5r6r7r8r9r:)Arrrfr!rr rrrr rrErr\r9rrrrGrr^Zlogspacelog10rrr timerHrPcatch_warnings simplefilterrrrrrzipextendsortedoperator itemgetter enumeraterrr:rrLlenrNrOr rarrayZ cv_results_ZstdZalpha_r[r5r6r7r8rTr'rirj)ryrrrr.rpathZn_alphasr(rZalpha_1Zalpha_0Z routed_paramst0rVZ this_pathZcovsrSZscoresZ best_scoreZlast_finite_idxindexrZ best_indexZ grid_scoresZ best_alphar+rr,rhs   $   (              " zGraphicalLassoCV.fitcCs.t|jjdjt|jtjdddd}|S)ajGet metadata routing of this object. Please check :ref:`User Guide ` on how the routing mechanism works. .. versionadded:: 1.5 Returns ------- routing : MetadataRouter A :class:`~sklearn.utils.metadata_routing.MetadataRouter` encapsulating routing information. )ownerrrh)ZcalleeZcaller)rZmethod_mapping)rr{r|addr rr)ryZrouterr+r+r,get_metadata_routingbs z%GraphicalLassoCV.get_metadata_routing)N)r|r}r~rrorur rrFrr!rrr:rxrrhrrr+r+rzr,rs4 7   ;r);rrrrrPnumbersrrnumpyr!Zscipyrbaser exceptionsrZ linear_modelrrIr Zmodel_selectionr r utilsr Zutils._param_validationr rrZutils.metadata_routingrrrrrZutils.parallelrrZutils.validationrrrrrrrr-r0rrr:r[r^rnrorgrrr+r+r+r,sz          G