a jz@sdZddlmZmZddlZddlmZmZm Z m Z ddl m Z m Z mZddlmZddlmZmZmZmZmZdd lmZdd lmZmZdd lmZdd lmZm Z m!Z!dd l"m#Z#ddl$m%Z%m&Z&m'Z'd.ddZ(d/ddZ)d0ddZ*dddddddddd dd Z+e!d!egeeddd"d#geeddd"d#geeddd"d#ge hd$geeddd"d#geeddd"d#ge hd%geeddd"d#geeddd"d#gd&gdegd' d(d)dddddddddd d*d+Z,Gd,d-d-eeeeZ-dS)1zLocally Linear Embedding)IntegralRealN)eighqrsolvesvd) csr_matrixeye lil_matrix)eigsh) BaseEstimatorClassNamePrefixFeaturesOutMixinTransformerMixin _fit_context_UnstableArchMixin)NearestNeighbors) check_arraycheck_random_state)_init_arpack_v0)Interval StrOptionsvalidate_params) stable_cumsum) FLOAT_DTYPEScheck_is_fitted validate_dataMbP?cCst|td}t|td}t|td}|j\}}|jd|ks@Jtj||f|jd}tj||jd}t|D]\}} || } | ||} t | | j } t | } | dkr|| }n|}| j dd|d|7<t | |dd}|t|||ddf<ql|S)aCompute barycenter weights of X from Y along the first axis We estimate the weights to assign to each point in Y[indices] to recover the point X[i]. The barycenter weights sum to 1. Parameters ---------- X : array-like, shape (n_samples, n_dim) Y : array-like, shape (n_samples, n_dim) indices : array-like, shape (n_samples, n_dim) Indices of the points in Y used to compute the barycenter reg : float, default=1e-3 Amount of regularization to add for the problem to be well-posed in the case of n_neighbors > n_dim Returns ------- B : array-like, shape (n_samples, n_neighbors) Notes ----- See developers note for more information. dtyperNpos)Zassume_a)rrintshapenpemptyrones enumeratedotTtraceflatrsum)XYindicesreg n_samples n_neighborsBviindACGr*Rwr<FE:\soccer\ml_env\Lib\site-packages\sklearn/manifold/_locally_linear.pybarycenter_weightss&       r>c Cst|d|d|}|j}|j}|j|ddddddf}t||||d}td||d|}t| | |f||fdS) a-Computes the barycenter weighted graph of k-Neighbors for points in X Parameters ---------- X : {array-like, NearestNeighbors} Sample data, shape = (n_samples, n_features), in the form of a numpy array or a NearestNeighbors object. n_neighbors : int Number of neighbors for each sample. reg : float, default=1e-3 Amount of regularization when solving the least-squares problem. Only relevant if mode='barycenter'. If None, use the default. n_jobs : int or None, default=None The number of parallel jobs to run for neighbors search. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. Returns ------- A : sparse matrix in CSR format, shape = [n_samples, n_samples] A[i, j] is assigned the weight of edge that connects i to j. See Also -------- sklearn.neighbors.kneighbors_graph sklearn.neighbors.radius_neighbors_graph r r2n_jobsF)return_distanceNr0rr#) rfit_fit_XZn_samples_fit_ kneighborsr>r$ZarangerZravel) r-r2r0r@Zknnr1r6dataZindptrr<r<r=barycenter_kneighbors_graphRs!rHr arpackư>dc Cs2|dkr,|jddkr(||dkr(d}nd}|dkrt|jd|}z t|||d|||d\}} Wn2ty} ztd | | WYd } ~ n d } ~ 00| d d |d ft||d fS|dkr"t|d r|}t ||||d fd d\}} t t |} | d d | ft|fStd|d S)a0 Find the null space of a matrix M. Parameters ---------- M : {array, matrix, sparse matrix, LinearOperator} Input covariance matrix: should be symmetric positive semi-definite k : int Number of eigenvalues/vectors to return k_skip : int, default=1 Number of low eigenvalues to skip. eigen_solver : {'auto', 'arpack', 'dense'}, default='arpack' auto : algorithm will attempt to choose the best method for input data arpack : use arnoldi iteration in shift-invert mode. For this method, M may be a dense matrix, sparse matrix, or general linear operator. Warning: ARPACK can be unstable for some problems. It is best to try several random seeds in order to check results. dense : use standard dense matrix operations for the eigenvalue decomposition. For this method, M must be an array or matrix type. This method should be avoided for large problems. tol : float, default=1e-6 Tolerance for 'arpack' method. Not used if eigen_solver=='dense'. max_iter : int, default=100 Maximum number of iterations for 'arpack' method. Not used if eigen_solver=='dense' random_state : int, RandomState instance, default=None Determines the random number generator when ``solver`` == 'arpack'. Pass an int for reproducible results across multiple function calls. See :term:`Glossary `. autor rIdenseg)sigmatolmaxiterv0a Error in determining null-space with ARPACK. Error message: '%s'. Note that eigen_solver='arpack' can fail when the weight matrix is singular or otherwise ill-behaved. In that case, eigen_solver='dense' is recommended. See online documentation for more information.Ntoarrayr T)Zsubset_by_indexZ overwrite_azUnrecognized eigen_solver '%s') r#rr RuntimeError ValueErrorr$r,hasattrrTrZargsortabs) Mkk_skip eigen_solverrQmax_iter random_staterSZ eigen_valuesZ eigen_vectorseindexr<r<r= null_space|s8*&   rarLstandard-C6?-q=) r0r\rQr]method hessian_tol modified_tolr^r@c < Cst|d| d} | || j}|j\} }||kr:td|| krRtd| |f|dk}|rbtntj}|dkrt| ||| d}|rt |jd|j i|}|j |}n:|j ||j | }|j dd|jd dd7<n|d kr||dd }|||krtd | j||dd d}|ddddf}tj|d||ftjd}d|ddd f<|| | ftjd}||k}t| D]v}|||}||d 8}|rt|d dd }n,t||j }t|ddddddf}|ddd|f|dddd|f<d|}t|D]V}|dd||df|dd||f|dd||||f<|||7}q&t|\}}|dd|ddf}|d } d| tt| |k<|| }t||||\}!}"||!|"ft||j 7<qn|dkr(||kr$td| j||dd d}|ddddf}t| ||f}#t||}$t| |$g}%||k}|rt| D]4}|||||}&t|&dd\|#|<|%|<}'q|%d C}%njt| D]`}|||||}&t|&|&j }(t|(\})}*|)ddd|%|<|*dddddf|#|<qd|%d}t|#d d dt|}+|+ddd|$f|%|dddf<|+dd|$df|dddf<t| |f},t| D]}t|#||+||,|<q|,|,ddddf},|%dd|dfd|%ddd|fd}-t|-}.tj| td}/t |%d}0|0ddddf|0ddddfd}1t| D]$}t!|1|dddf|.|/|<q|/||$7}/|| | ftjd}t| D]H}|/|}2|#|dd||2df}3tj"#|3d t$|2}4t%|2|4t|3j t|}5tj"#|5}6|6| kr\|5d 9}5n|5|6}5|3d t&t|3|5|5d|4|,|dddf}7t||||\}!}"||!|"ft|7|7j 7<|7d}8||||f|88<||||gf|88<|||f|27<qؐnj|dkr| j||dd d}|ddddf}|| | ftjd}||k}t| D]}|||}9|9|9d 8}9|rt|9ddd }:n,t|9|9j }t|ddddddf}:t||df}|:ddd|f|ddddf<dt$||ddd f<t||j };t||||\}!}"||!|"f|;8<|||||ftj|d7<q||r|'}t(||d|||| dS)Nr r?z>output dimension must be less than or equal to input dimensionzHExpected n_neighbors <= n_samples, but n_samples = %d, n_neighbors = %drOrb)r2r0r@formatrhessianr z^for method='hessian', n_neighbors must be greater than [n_components * (n_components + 3) / 2]Fr2rAr)Z full_matricesmodifiedz1modified LLE requires n_neighbors >= n_componentsTrltsag?rC)r[r\rQr]r^))rrDrEr#rVr r$ZzerosrHr rhr)rTr+rFr%Zfloat64rangemeanrr(rrr,whererXZmeshgridminZ transposer&Zmedianr"rZ searchsortedZlinalgZnormsqrtfullouterZtocsrra)rIrLrO>rlrirbrmr^ r-r2rur0r\rQr]rerfrgr^r@TZprefer_skip_nested_validationc Cs t|||||||||| | | d S)aPerform a Locally Linear Embedding analysis on the data. Read more in the :ref:`User Guide `. Parameters ---------- X : {array-like, NearestNeighbors} Sample data, shape = (n_samples, n_features), in the form of a numpy array or a NearestNeighbors object. n_neighbors : int Number of neighbors to consider for each point. n_components : int Number of coordinates for the manifold. reg : float, default=1e-3 Regularization constant, multiplies the trace of the local covariance matrix of the distances. eigen_solver : {'auto', 'arpack', 'dense'}, default='auto' auto : algorithm will attempt to choose the best method for input data arpack : use arnoldi iteration in shift-invert mode. For this method, M may be a dense matrix, sparse matrix, or general linear operator. Warning: ARPACK can be unstable for some problems. It is best to try several random seeds in order to check results. dense : use standard dense matrix operations for the eigenvalue decomposition. For this method, M must be an array or matrix type. This method should be avoided for large problems. tol : float, default=1e-6 Tolerance for 'arpack' method Not used if eigen_solver=='dense'. max_iter : int, default=100 Maximum number of iterations for the arpack solver. method : {'standard', 'hessian', 'modified', 'ltsa'}, default='standard' standard : use the standard locally linear embedding algorithm. see reference [1]_ hessian : use the Hessian eigenmap method. This method requires n_neighbors > n_components * (1 + (n_components + 1) / 2. see reference [2]_ modified : use the modified locally linear embedding algorithm. see reference [3]_ ltsa : use local tangent space alignment algorithm see reference [4]_ hessian_tol : float, default=1e-4 Tolerance for Hessian eigenmapping method. Only used if method == 'hessian'. modified_tol : float, default=1e-12 Tolerance for modified LLE method. Only used if method == 'modified'. random_state : int, RandomState instance, default=None Determines the random number generator when ``solver`` == 'arpack'. Pass an int for reproducible results across multiple function calls. See :term:`Glossary `. n_jobs : int or None, default=None The number of parallel jobs to run for neighbors search. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. Returns ------- Y : ndarray of shape (n_samples, n_components) Embedding vectors. squared_error : float Reconstruction error for the embedding vectors. Equivalent to ``norm(Y - W Y, 'fro')**2``, where W are the reconstruction weights. References ---------- .. [1] Roweis, S. & Saul, L. Nonlinear dimensionality reduction by locally linear embedding. Science 290:2323 (2000). .. [2] Donoho, D. & Grimes, C. Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data. Proc Natl Acad Sci U S A. 100:5591 (2003). .. [3] `Zhang, Z. & Wang, J. MLLE: Modified Locally Linear Embedding Using Multiple Weights. `_ .. [4] Zhang, Z. & Zha, H. Principal manifolds and nonlinear dimensionality reduction via tangent space alignment. Journal of Shanghai Univ. 8:406 (2004) Examples -------- >>> from sklearn.datasets import load_digits >>> from sklearn.manifold import locally_linear_embedding >>> X, _ = load_digits(return_X_y=True) >>> X.shape (1797, 64) >>> embedding, _ = locally_linear_embedding(X[:100],n_neighbors=5, n_components=2) >>> embedding.shape (100, 2) r)rrr<r<r=locally_linear_embeddings rc@seZdZUdZeeddddgeeddddgeeddddgehdgeeddddgeeddddgehdgeeddddgeeddddgehd gd gdegd Ze e d <d dddddddddddd ddZ ddZ e ddd"ddZe ddd#ddZd d!ZdS)$LocallyLinearEmbeddingaLocally Linear Embedding. Read more in the :ref:`User Guide `. Parameters ---------- n_neighbors : int, default=5 Number of neighbors to consider for each point. n_components : int, default=2 Number of coordinates for the manifold. reg : float, default=1e-3 Regularization constant, multiplies the trace of the local covariance matrix of the distances. eigen_solver : {'auto', 'arpack', 'dense'}, default='auto' The solver used to compute the eigenvectors. The available options are: - `'auto'` : algorithm will attempt to choose the best method for input data. - `'arpack'` : use arnoldi iteration in shift-invert mode. For this method, M may be a dense matrix, sparse matrix, or general linear operator. - `'dense'` : use standard dense matrix operations for the eigenvalue decomposition. For this method, M must be an array or matrix type. This method should be avoided for large problems. .. warning:: ARPACK can be unstable for some problems. It is best to try several random seeds in order to check results. tol : float, default=1e-6 Tolerance for 'arpack' method Not used if eigen_solver=='dense'. max_iter : int, default=100 Maximum number of iterations for the arpack solver. Not used if eigen_solver=='dense'. method : {'standard', 'hessian', 'modified', 'ltsa'}, default='standard' - `standard`: use the standard locally linear embedding algorithm. see reference [1]_ - `hessian`: use the Hessian eigenmap method. This method requires ``n_neighbors > n_components * (1 + (n_components + 1) / 2``. see reference [2]_ - `modified`: use the modified locally linear embedding algorithm. see reference [3]_ - `ltsa`: use local tangent space alignment algorithm. see reference [4]_ hessian_tol : float, default=1e-4 Tolerance for Hessian eigenmapping method. Only used if ``method == 'hessian'``. modified_tol : float, default=1e-12 Tolerance for modified LLE method. Only used if ``method == 'modified'``. neighbors_algorithm : {'auto', 'brute', 'kd_tree', 'ball_tree'}, default='auto' Algorithm to use for nearest neighbors search, passed to :class:`~sklearn.neighbors.NearestNeighbors` instance. random_state : int, RandomState instance, default=None Determines the random number generator when ``eigen_solver`` == 'arpack'. Pass an int for reproducible results across multiple function calls. See :term:`Glossary `. n_jobs : int or None, default=None The number of parallel jobs to run. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. Attributes ---------- embedding_ : array-like, shape [n_samples, n_components] Stores the embedding vectors reconstruction_error_ : float Reconstruction error associated with `embedding_` n_features_in_ : int Number of features seen during :term:`fit`. .. versionadded:: 0.24 feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 nbrs_ : NearestNeighbors object Stores nearest neighbors instance, including BallTree or KDtree if applicable. See Also -------- SpectralEmbedding : Spectral embedding for non-linear dimensionality reduction. TSNE : Distributed Stochastic Neighbor Embedding. References ---------- .. [1] Roweis, S. & Saul, L. Nonlinear dimensionality reduction by locally linear embedding. Science 290:2323 (2000). .. [2] Donoho, D. & Grimes, C. Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data. Proc Natl Acad Sci U S A. 100:5591 (2003). .. [3] `Zhang, Z. & Wang, J. MLLE: Modified Locally Linear Embedding Using Multiple Weights. `_ .. [4] Zhang, Z. & Zha, H. Principal manifolds and nonlinear dimensionality reduction via tangent space alignment. Journal of Shanghai Univ. 8:406 (2004) Examples -------- >>> from sklearn.datasets import load_digits >>> from sklearn.manifold import LocallyLinearEmbedding >>> X, _ = load_digits(return_X_y=True) >>> X.shape (1797, 64) >>> embedding = LocallyLinearEmbedding(n_components=2) >>> X_transformed = embedding.fit_transform(X[:100]) >>> X_transformed.shape (100, 2) r Nrrr>rIrLrO>rlrirbrm>Zkd_treeZ ball_treeZbruterLr^) r2rur0r\rQr]rerfrgneighbors_algorithmr^r@_parameter_constraintsr rrLrJrKrbrcrdc CsL||_||_||_||_||_||_||_||_| |_| |_ | |_ | |_ dS)N) r2rur0r\rQr]rerfrgr^rr@) selfr2rur0r\rQr]rerfrgrr^r@r<r<r=__init__szLocallyLinearEmbedding.__init__cCst|j|j|jd|_t|j}t||td}|j |t |j|j|j |j |j |j|j|j|j||j|jd \|_|_|jjd|_dS)N)r2 algorithmr@r) r-r2rur\rQr]rerfrgr^r0r@r )rr2rr@nbrs_rr^rfloatrDrrur\rQr]rerfrgr0 embedding_Zreconstruction_error_r#Z_n_features_out)rr-r^r<r<r=_fit_transforms.  z%LocallyLinearEmbedding._fit_transformTrcCs|||S)ayCompute the embedding vectors for data X. Parameters ---------- X : array-like of shape (n_samples, n_features) Training set. y : Ignored Not used, present here for API consistency by convention. Returns ------- self : object Fitted `LocallyLinearEmbedding` class instance. )rrr-yr<r<r=rD*s zLocallyLinearEmbedding.fitcCs|||jS)aCompute the embedding vectors for data X and transform X. Parameters ---------- X : array-like of shape (n_samples, n_features) Training set. y : Ignored Not used, present here for API consistency by convention. Returns ------- X_new : array-like, shape (n_samples, n_components) Returns the instance itself. )rrrr<r<r= fit_transform>s z$LocallyLinearEmbedding.fit_transformcCst|t||dd}|jj||jdd}t||jj||jd}t |j d|j f}t |j dD]$}t |j||j||||<qd|S)a Transform new points into embedding space. Parameters ---------- X : array-like of shape (n_samples, n_features) Training set. Returns ------- X_new : ndarray of shape (n_samples, n_components) Returns the instance itself. Notes ----- Because of scaling performed by this method, it is discouraged to use it together with methods that are not scale-invariant (like SVMs). F)resetrjrBr)rrrrFr2r>rEr0r$r%r#rurnr(rr))rr-r6weightsZX_newr5r<r<r= transformRs"z LocallyLinearEmbedding.transform)N)N)__name__ __module__ __qualname____doc__rrrrrdict__annotations__rrrrDrrr<r<r<r=rZsD       r)r)rN)r rIrJrKN).rnumbersrrnumpyr$Z scipy.linalgrrrrZ scipy.sparserr r Zscipy.sparse.linalgr baser rrrrrxrutilsrrZ utils._arpackrZutils._param_validationrrrZ utils.extmathrZutils.validationrrrr>rHrarrrr<r<r<r=sx     6 + Q y