Class DynamicDataAnalyzerImpl

java.lang.Object
dataAnalysis.DynamicDataAnalyzerImpl
All Implemented Interfaces:
DynamicDataAnalyzer
Direct Known Subclasses:
QuantileDataAnalyzer
public class DynamicDataAnalyzerImpl extends Object implements DynamicDataAnalyzer
Dynamic data analyzer of the JSIM engine. It removes the initial transient of the measured samples using the R5 heuristic [Fis73] and the MSER-5 stationarity rule [Spr98], and then generates the confidence intervals with the spectral method [HW81], as described in the JSIM chapters of the JMT user manual.

In detail, this class can remove the initial transient period and performs calculation of variance and confidence interval of the sample mean of a sequence.
It is robust to data correlation.

See:
(1) K. Pawlikowski, "Steady­State Simulation of Queueing Processes: A Survey of Problems and Solutions", ACM Computing Surveys 22, 2 (1990), pp. 122--170

Version:
Modified 7/02/2003
Author:
Federico Granata, Bertoli Marco - Bugfixing statistical part (R5 and Schruben) 18-11-2005

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    protected double
    alfa
    the quantile required for the confidence interval
    protected int
    batchLen
    number of samples per batch
    protected double[]
    batchMean
    mean of batches
    protected double
    C1
    constant chosen to reduce bias of variance estimator
    protected int
    C2
    degrees of freedom of the t-distribution
    protected double
    confInt
    the estimated confidence intervals
    protected boolean
    deleted
    Transient period has been deleted or not
    protected int
    delta
     
    protected int
    discarded
    number of discarded data in deleting transient period
    protected boolean
    end
    end = true if the analysis has been completed (successfully or not)
    int
    eurDisc
    First estimate of transient length: this number of samples is identified by the heuristic test
    protected double
    extMean
    the estimated mean
    protected double
    extVar
    the estimated variance
    protected boolean
    heuristic
    heuristic = true if the heuristic rule for startup period is successful.
    protected DoubleArrayList
    initData
    starting data, not steady state
    protected int
    initial
    number of data that are replaced when detect initial test fails.
    protected boolean
    initialized
    initialized = true if the system is in steady state
    protected DoubleArrayList
    initWeight
    starting weights, not steady state
    protected int
    K
    number of points of the polyfit
    protected int
    maxData
    maximum number of data to be collected
    protected int
    nSamples
    actual number of samples
    protected int
    numBatch
    number of batches
    protected SimParameters
    parameters
     
    protected int
    polyOrder
    the order of the polynomial
    protected double
    precision
    maximum amplitude of confidence interval
    protected boolean
    success
    success = true if the analysis is successful, the required precision has been met
    protected double
    sumLastBatch
    the sum of samples of the last batch
    protected double[]
    weightBatchMean
    mean of batches weighted

  • Constructor Summary

    Constructors
    Constructor
    Description
    DynamicDataAnalyzerImpl(double alfa, double precision, int maxData)
    Creates a new DynamicDataAnalyzer

  • Method Summary

    Modifier and Type
    Method
    Description
    boolean
    addSample(double newSample, double weight)
    Adds the new sample to the statistic.
    protected void
    calcConfInterval()
    updates the conf intervals
    double[]
    calcConstants(int K, int polyOrder)
    computes the degrees of freedom of the tStudent distribution used in the estimation of variance and confidence intervals C2, and a constant used to depolarize the variance estimator
    protected double
    calcEstimatorVar(int K, int polyOrder)
    computes the variance of the estimator of the variance
    protected void
    calcVar()
    updates the variance
    double
    calcVar(double[] data, int start, int n, int K, int polyOrder)
    calculates the variance through Heidelberger & Welch method, if the data are the means of batches of size batchLen, the variance must be multiplied by batchLen.
    protected boolean
    deleteTransient()
    Apply the heuristic rule MSR5 to discard the most biased samples of the time series.
    double
    estimatedMean()
    Gets the estimated mean, even if the confidence requirements have not been reached
    double
    getAlfa()
    gets the actual requested alpha of the analyzer
    double
    getConfInt()
    gets the half width confidence interval estimated of the monitored data stream.
    double[]
    getData()
    Gets the batch means.
    int
    getDiscarded()
    gets the number of discarded data.
    double
    getLastIntervalAvg()
    This method return inverse of the instantaneous value
    int
    getMaxData()
    gets the maximum number of samples that can be analyzed.
    double
    getMean()
    gets the mean estimated of the monitored data stream
    double
    getNullMeasure_upperLimit()
    This method should be called only if mean is equal to zero (@see isZero()), that is when nullTest() returns true.
    double
    getPrecision()
    gets the actual requested precision of the analyzer
    int
    getSamples()
    number of samples elaborated up to now
    boolean
    getSuccess()
    true if the analysis is successful
    double
    getVar()
    gets the variance estimated of the monitored data stream
    protected boolean
    heuristicR5(double newSample, double Weight)
    It uses the heuristic rule R5.
    protected boolean
    HWtest()
    Applies the spectral test to generate the Confidence Intervals.
    boolean
    isZero()
    Returns true if null test has found out that this measure is zero
    protected boolean
    nullTest()
    Verifies whether the measure is almost always equal to zero
    protected boolean
    schrubenTest(double newSample, double Weight)
    inserts a new value of the sequence, and detects when the initial transient period has ended.
    void
    setParameters(SimParameters parameters)
    Sets the reference to the class containing simulation parameters.
    void
    stopMeasure(boolean success)
    Stops the measuring process
    String
    toString()
    creates a string of the principals parameters calculated

    Methods inherited from class Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

  • Field Details

    • parameters

      protected SimParameters parameters

    • numBatch

      protected int numBatch
      number of batches

    • batchLen

      protected int batchLen
      number of samples per batch

    • batchMean

      protected double[] batchMean
      mean of batches

    • weightBatchMean

      protected double[] weightBatchMean
      mean of batches weighted

    • extMean

      protected double extMean
      the estimated mean

    • extVar

      protected double extVar
      the estimated variance

    • confInt

      protected double confInt
      the estimated confidence intervals

    • sumLastBatch

      protected double sumLastBatch
      the sum of samples of the last batch

    • nSamples

      protected int nSamples
      actual number of samples

    • maxData

      protected int maxData
      maximum number of data to be collected

    • C1

      protected double C1
      constant chosen to reduce bias of variance estimator

    • C2

      protected int C2
      degrees of freedom of the t-distribution

    • alfa

      protected double alfa
      the quantile required for the confidence interval

    • precision

      protected double precision
      maximum amplitude of confidence interval

    • initData

      protected DoubleArrayList initData
      starting data, not steady state

    • initWeight

      protected DoubleArrayList initWeight
      starting weights, not steady state

    • deleted

      protected boolean deleted
      Transient period has been deleted or not

    • initialized

      protected boolean initialized
      initialized = true if the system is in steady state

    • end

      protected boolean end
      end = true if the analysis has been completed (successfully or not)

    • success

      protected boolean success
      success = true if the analysis is successful, the required precision has been met

    • heuristic

      protected boolean heuristic
      heuristic = true if the heuristic rule for startup period is successful.

    • discarded

      protected int discarded
      number of discarded data in deleting transient period

    • initial

      protected int initial
      number of data that are replaced when detect initial test fails.

    • delta

      protected int delta

    • K

      protected int K
      number of points of the polyfit

    • polyOrder

      protected int polyOrder
      the order of the polynomial

    • eurDisc

      public int eurDisc
      First estimate of transient length: this number of samples is identified by the heuristic test

  • Constructor Details

    • DynamicDataAnalyzerImpl

      public DynamicDataAnalyzerImpl(double alfa, double precision, int maxData)
      Creates a new DynamicDataAnalyzer
      Parameters:
      alfa - the quantile required for the confidence interval
      precision - maximum amplitude of confidence interval (precision = maxamplitude / mean )
      maxData - maximum number of data to be analyzed (at least 5000)

  • Method Details

    • addSample

      public boolean addSample(double newSample, double weight)
      Adds the new sample to the statistic.
      Specified by:
      addSample in interface DynamicDataAnalyzer
      Parameters:
      newSample - the new sample
      weight - the weight of the newSample, if it is not needed put 1.
      Returns:
      true if the computation of confidence interval has finished (i.e. if the confidence interval is smaller than the one required by the user, or if the analyzed data are too many), false otherwise

    • heuristicR5

      protected boolean heuristicR5(double newSample, double Weight)
      It uses the heuristic rule R5. Inserts a new value of the sequence, and approximately detects the initial transient period.
      See
      K. Pawlikowski "Steady­State Simulation of Queueing Processes: A Survey of Problems and Solutions" ACM Computing Surveys 22, 2 (1990), pp. 122--170.

    • schrubenTest

      protected boolean schrubenTest(double newSample, double Weight)
      inserts a new value of the sequence, and detects when the initial transient period has ended.

    • deleteTransient

      protected boolean deleteTransient()
      Apply the heuristic rule MSR5 to discard the most biased samples of the time series. It minimizes the Mean Square Error.

    • HWtest

      protected boolean HWtest()
      Applies the spectral test to generate the Confidence Intervals.
      See:
      P. Heidelberger,Peter D. Welch
      "A spectral method for confidence interval generation and run length control in simulations"
      Returns:
      true if the precision requirement is reached. false if not.

    • calcVar

      protected void calcVar()
      updates the variance

    • toString

      public String toString()
      creates a string of the principals parameters calculated
      Overrides:
      toString in class Object
      Returns:
      the string

    • getSamples

      public int getSamples()
      number of samples elaborated up to now
      Specified by:
      getSamples in interface DynamicDataAnalyzer
      Returns:
      number of samples

    • getMean

      public double getMean()
      gets the mean estimated of the monitored data stream
      Specified by:
      getMean in interface DynamicDataAnalyzer
      Returns:
      the mean

    • getVar

      public double getVar()
      gets the variance estimated of the monitored data stream
      Returns:
      the variance

    • getConfInt

      public double getConfInt()
      gets the half width confidence interval estimated of the monitored data stream.
      Specified by:
      getConfInt in interface DynamicDataAnalyzer
      Returns:
      the confidence interval requested

    • getSuccess

      public boolean getSuccess()
      true if the analysis is successful
      Specified by:
      getSuccess in interface DynamicDataAnalyzer
      Returns:
      true if the analysis respect all'users requests

    • calcVar

      public double calcVar(double[] data, int start, int n, int K, int polyOrder)
      calculates the variance through Heidelberger & Welch method, if the data are the means of batches of size batchLen, the variance must be multiplied by batchLen.
      Parameters:
      data - array containing the sequence of data
      start - the starting sample
      n - number of data of the sequence to be estimated
      K - number of points used in the polynomial fitting must be <= n/4, typically is 25, 50
      polyOrder - the order of the polynomial to be fitted to the getLog of the spectrum of data typically is 2 or 3.
      Returns:
      the estimated variance

    • calcConstants

      public double[] calcConstants(int K, int polyOrder)
      computes the degrees of freedom of the tStudent distribution used in the estimation of variance and confidence intervals C2, and a constant used to depolarize the variance estimator
      Parameters:
      K - number of points of the polyfit
      polyOrder - the order of the polynomial
      Returns:
      the double array of dimension 2, at first position it is C1 and at second C2

    • calcEstimatorVar

      protected double calcEstimatorVar(int K, int polyOrder)
      computes the variance of the estimator of the variance

    • calcConfInterval

      protected void calcConfInterval()
      updates the conf intervals

    • getPrecision

      public double getPrecision()
      gets the actual requested precision of the analyzer
      Specified by:
      getPrecision in interface DynamicDataAnalyzer
      Returns:
      the precision

    • getAlfa

      public double getAlfa()
      gets the actual requested alpha of the analyzer
      Specified by:
      getAlfa in interface DynamicDataAnalyzer
      Returns:
      the alpha

    • getMaxData

      public int getMaxData()
      gets the maximum number of samples that can be analyzed.
      Specified by:
      getMaxData in interface DynamicDataAnalyzer
      Returns:
      maxData

    • getDiscarded

      public int getDiscarded()
      gets the number of discarded data.
      Specified by:
      getDiscarded in interface DynamicDataAnalyzer
      Returns:
      discarded data

    • getData

      public double[] getData()
      Gets the batch means. If the data is not over the transient period returns null.
      Returns:
      batch means

    • nullTest

      protected boolean nullTest()
      Verifies whether the measure is almost always equal to zero
      Returns:
      true if the measure is almost always equal to zero

    • isZero

      public boolean isZero()
      Returns true if null test has found out that this measure is zero
      Specified by:
      isZero in interface DynamicDataAnalyzer
      Returns:
      true if null test has found out that this measure is zero

    • getNullMeasure_upperLimit

      public double getNullMeasure_upperLimit()
      This method should be called only if mean is equal to zero (@see isZero()), that is when nullTest() returns true.
      Specified by:
      getNullMeasure_upperLimit in interface DynamicDataAnalyzer
      Returns:
      the upper limit of confidence interval when mean is equal to 0.0

    • estimatedMean

      public double estimatedMean()
      Gets the estimated mean, even if the confidence requirements have not been reached
      Specified by:
      estimatedMean in interface DynamicDataAnalyzer
      Returns:
      the estimated mean

    • stopMeasure

      public void stopMeasure(boolean success)
      Stops the measuring process
      Specified by:
      stopMeasure in interface DynamicDataAnalyzer
      Parameters:
      success - value to be reported by getSuccess()

    • setParameters

      public void setParameters(SimParameters parameters)
      Sets the reference to the class containing simulation parameters. The parameters are updated with the passed values.
      Specified by:
      setParameters in interface DynamicDataAnalyzer
      Parameters:
      parameters -

    • getLastIntervalAvg

      public double getLastIntervalAvg()
      Description copied from interface: DynamicDataAnalyzer
      This method return inverse of the instantaneous value
      Specified by:
      getLastIntervalAvg in interface DynamicDataAnalyzer