a j;8@sfdZddlZddlZddlZddlmZddlmZmZddl m Z m Z ddl Z ddl ZddlmZmZddlmZdd lmZmZmZd d lmZd d lmZmZmZmZed dddZeddddZ e!e"Z#eehddge$ej%dgdgdgdgdgdgdgeed dddgeeddddgd dddddddddddd d d!d"Z&d'd#d$Z'd%d&Z(dS)(zKDDCUP 99 dataset. A classic dataset for anomaly detection. The dataset page is available from UCI Machine Learning Repository https://archive.ics.uci.edu/ml/machine-learning-databases/kddcup99-mld/kddcup.data.gz N)GzipFile)IntegralReal)existsjoin)Bunchcheck_random_state)shuffle)Interval StrOptionsvalidate_params) get_data_home)RemoteFileMetadata_convert_data_dataframe _fetch_remote load_descrZ kddcup99_dataz.https://ndownloader.figshare.com/files/5976045Z@3b6c942aa0356c0ca35b7b595a26c89d343652c9db428893e7494f837b274292)filenameurlZchecksumZkddcup99_10_dataz.https://ndownloader.figshare.com/files/5976042Z@8045aca0d84e70e622d1148d7df782496f6333bf6eb979a1b0837c42a9fd9561>SAsmtphttpSFboolean random_stateleft)closedgZneither) subset data_homer r percent10download_if_missing return_X_yas_frame n_retriesdelayT)Zprefer_skip_nested_validationF?c Cst|d}t||||| d} | j} | j} | j} | j}|dkr| dk}t|}| |ddf}| |}| |ddf}| |}|jd}t |}| d|d}||}||}tj ||f} tj ||f} |dks|d ks|d kr&| ddd fd k}tj | |dd f| |d dff} | dd | d d} | |} t | dddfdjtdd| dddf<t | dddfdjtdd| dddf<t | dddfdjtdd| dddf<|d krF| dddfdk}| |} | |} tj | dddf| dddf| dddff} | d| d| dg} |d kr| dddfdk}| |} | |} tj | dddf| dddf| dddff} | d| d| dg} |dkr&tj | dddf| dddf| dddf| dddff} | d| d| d| dg} |r>t| | |d\} } td}d}|rftd| | | |\}} } |rt| | fSt| | ||| |dS)a Load the kddcup99 dataset (classification). Download it if necessary. ================= ==================================== Classes 23 Samples total 4898431 Dimensionality 41 Features discrete (int) or continuous (float) ================= ==================================== Read more in the :ref:`User Guide `. .. versionadded:: 0.18 Parameters ---------- subset : {'SA', 'SF', 'http', 'smtp'}, default=None To return the corresponding classical subsets of kddcup 99. If None, return the entire kddcup 99 dataset. data_home : str or path-like, default=None Specify another download and cache folder for the datasets. By default all scikit-learn data is stored in '~/scikit_learn_data' subfolders. .. versionadded:: 0.19 shuffle : bool, default=False Whether to shuffle dataset. random_state : int, RandomState instance or None, default=None Determines random number generation for dataset shuffling and for selection of abnormal samples if `subset='SA'`. Pass an int for reproducible output across multiple function calls. See :term:`Glossary `. percent10 : bool, default=True Whether to load only 10 percent of the data. download_if_missing : bool, default=True If False, raise an OSError if the data is not locally available instead of trying to download the data from the source site. return_X_y : bool, default=False If True, returns ``(data, target)`` instead of a Bunch object. See below for more information about the `data` and `target` object. .. versionadded:: 0.20 as_frame : bool, default=False If `True`, returns a pandas Dataframe for the ``data`` and ``target`` objects in the `Bunch` returned object; `Bunch` return object will also have a ``frame`` member. .. versionadded:: 0.24 n_retries : int, default=3 Number of retries when HTTP errors are encountered. .. versionadded:: 1.5 delay : float, default=1.0 Number of seconds between retries. .. versionadded:: 1.5 Returns ------- data : :class:`~sklearn.utils.Bunch` Dictionary-like object, with the following attributes. data : {ndarray, dataframe} of shape (494021, 41) The data matrix to learn. If `as_frame=True`, `data` will be a pandas DataFrame. target : {ndarray, series} of shape (494021,) The regression target for each sample. If `as_frame=True`, `target` will be a pandas Series. frame : dataframe of shape (494021, 42) Only present when `as_frame=True`. Contains `data` and `target`. DESCR : str The full description of the dataset. feature_names : list The names of the dataset columns target_names: list The names of the target columns (data, target) : tuple if ``return_X_y`` is True A tuple of two ndarray. The first containing a 2D array of shape (n_samples, n_features) with each row representing one sample and each column representing the features. The second ndarray of shape (n_samples,) containing the target samples. .. versionadded:: 0.20 r)rr r!r$r%rsnormal.Nri1 rrr r g?F)copyrshttpssmtp)rz kddcup99.rstfetch_kddcup99)datatargetframe target_names feature_namesZDESCR)r_fetch_brute_kddcup99r/r0r3r2npZ logical_notshaper randintZr_Zc_logastypefloatshuffle_methodrrr)rrr rr r!r"r#r$r%kddcup99r/r0r3r2stZnormal_samplesZnormal_targetsZabnormal_samplesZabnormal_targetsZn_samples_abnormalrZfdescrr1r@@E:\soccer\ml_env\Lib\site-packages\sklearn/datasets/_kddcup99.pyr.6sz   &000 4 4 B r.c*Cst|d}d}|r&t|d|}t}nt|d|}t}t|d}t|d} t|} dtfdd d d tfd tfd tfdtfdtfdtfdtfdtfdtfdtfdtfdtfdtfdtfdtfdtfdtfdtfdtfdtfdtfd tfd!tfd"tfd#tfd$tfd%tfd&tfd'tfd(tfd)tfd*tfd+tfd,tfd-tfd.tfd/tfd0g*} d1d2| D} | d3} | d4d3}| rzt|}t| }Wn<t y}z"t d5t |d6|WYd4}~n d4}~00n<|rt |t d7|jt||||d8t| }t d9t||j}t|d:d;}g}|D]&}|}||dq,|t d?t|tj|td@}t dAD],}|d4d4|f!|||d4d4|f<q|d4d4d4d3f}|d4d4d3f}tj"||dBdCtj"|| dBdCnt dDt#|||| gdES)Fa5Load the kddcup99 dataset, downloading it if necessary. Parameters ---------- data_home : str, default=None Specify another download and cache folder for the datasets. By default all scikit-learn data is stored in '~/scikit_learn_data' subfolders. download_if_missing : bool, default=True If False, raise an OSError if the data is not locally available instead of trying to download the data from the source site. percent10 : bool, default=True Whether to load only 10 percent of the data. n_retries : int, default=3 Number of retries when HTTP errors are encountered. delay : float, default=1.0 Number of seconds between retries. Returns ------- dataset : :class:`~sklearn.utils.Bunch` Dictionary-like object, with the following attributes. data : ndarray of shape (494021, 41) Each row corresponds to the 41 features in the dataset. target : ndarray of shape (494021,) Each value corresponds to one of the 21 attack types or to the label 'normal.'. feature_names : list The names of the dataset columns target_names: list The names of the target columns DESCR : str Description of the kddcup99 dataset. r(z-py3Z kddcup99_10r<Zsamplestargetsduration)Z protocol_typeZS4)ZserviceZS11)flagZS6Z src_bytesZ dst_bytesZlandZwrong_fragmentZurgentZhotZnum_failed_loginsZ logged_inZnum_compromisedZ root_shellZ su_attemptedZnum_rootZnum_file_creationsZ num_shellsZnum_access_filesZnum_outbound_cmdsZ is_host_loginZis_guest_logincountZ srv_countZ serror_rateZsrv_serror_rateZ rerror_rateZsrv_rerror_rateZ same_srv_rateZ diff_srv_rateZsrv_diff_host_rateZdst_host_countZdst_host_srv_countZdst_host_same_srv_rateZdst_host_diff_srv_rateZdst_host_same_src_port_rateZdst_host_srv_diff_host_rateZdst_host_serror_rateZdst_host_srv_serror_rateZdst_host_rerror_rateZdst_host_srv_rerror_rate)labelsZS16cSsg|] }|dqS)rr@).0cr@r@rA pz)_fetch_brute_kddcup99..Nz7The cache for fetch_kddcup99 is invalid, please delete z! and run the fetch_kddcup99 againzDownloading %s)dirnamer$r%zextracting archiver?)rmode ,zextraction done)dtype*r)compressz1Data not found and `download_if_missing` is False)r/r0r3r2)$rrARCHIVE_10_PERCENTARCHIVErintr:joblibload ExceptionOSErrorstr_mkdirploggerinforrr5rQdebugrr readlinesdecodeappendreplacesplitcloseosremoveZasarrayobjectranger9dumpr)rr!r r$r%Z dir_suffixZ kddcup_dirarchiveZ samples_pathZ targets_path availabledtZ column_namesr2r3XyeZDT archive_pathfile_ZXylinejr@r@rAr4 s+   -          *r4c CsFzt|Wn2ty@}z|jtjkr,WYd}~n d}~00dS)zgEnsure directory d exists (like mkdir -p on Unix) No guarantee that the directory is writable. N)rfmakedirsrZerrnoEEXIST)drpr@r@rAr\s  r\)NTTr&r'))__doc__rvloggingrfgziprnumbersrros.pathrrrWnumpyr5utilsrr r r;Zutils._param_validationr r r rOrZ_baserrrrrUrT getLogger__name__r]r[PathLiker.r4r\r@r@r@rAsl        G