zoo.models.anomalydetection package

Submodules

zoo.models.anomalydetection.anomaly_detector module

class zoo.models.anomalydetection.anomaly_detector.AnomalyDetector(feature_shape, hidden_layers=[8, 32, 15], dropouts=[0.2, 0.2, 0.2], **kwargs)

Bases: zoo.models.common.zoo_model.KerasZooModel

The anomaly detector model for sequence data based on LSTM.

# Arguments feature_shape: The input shape of features, including unroll_length and feature_size. hidden_layers: Units of hidden layers of LSTM. dropouts: Fraction of the input units to drop out. Float between 0 and 1.

build_model()

classmethod detect_anomalies(ytruth, ypredict, anomaly_size)

# Arguments :param ytruth: RDD of float or double values. Truth to be compared. :param ypredict: RDD of float or double values. Predictions. :param anomaly_size: Int. The size to be considered as anomalies. :return: RDD of [ytruth, ypredict, anomaly], anomaly is None or ytruth

static load_model(path, weight_path=None, bigdl_type='float')

Load an existing AnomalyDetector model (with weights).

# Arguments path: The path for the pre-defined model.

Local file system, HDFS and Amazon S3 are supported. HDFS path should be like ‘hdfs://[host]:[port]/xxx’. Amazon S3 path should be like ‘s3a://bucket/xxx’.

weight_path: The path for pre-trained weights if any. Default is None.

predict(x, batch_per_thread=8)

Precict on RDD[Sample].

static standardScale(df)

static train_test_split(unrolled, test_size)

classmethod unroll(data_rdd, unroll_length, predict_step=1)

Unroll a rdd of arrays to prepare features and labels.

# Arguments data_rdd: RDD[Array]. data to be unrolled, it holds original time series features unroll_length: Int. the length of precious values to predict future value. predict_step: Int. How many time steps to predict future value, default is 1. return: an rdd of FeatureLableIndex a simple example

data: (1,2,3,4,5,6); unrollLength: 2, predictStep: 1 features, label, index (1,2), 3, 0 (2,3), 4, 1 (3,4), 5, 2 (4,5), 6, 3

class zoo.models.anomalydetection.anomaly_detector.FeatureLableIndex(feature, label, index, bigdl_type='float')

Bases: object

Each record should contain the following fields: feature: List[List[float]]. label: float. index: long.

Module contents