Large speed-ups when the histograms have different bins (the default
simplify = FALSE), with results identical to 1.0.9 (up to
differences of the order of 1e-13 in covariances and fuzzy c-means, due
to the order of the sums). Times on a matrix of 300 histograms x 3
variables with different bins:
| Function | 1.0.9 | 1.0.10 |
|---|---|---|
WH_adaptive_fcmeans() |
117 s | 4.1 s |
WH_2d_Kohonen_maps() |
106 s | 4.6 s |
WH_2d_Adaptive_Kohonen_maps() |
101 s | 4.4 s |
WH.var.covar(), WH.correlation(),
WH.SSQ() |
4.8 s | 0.1 s |
WH.vec.mean() |
2.3 s | 0.08 s |
WH_fcmeans() |
7.8 s | 1.6 s |
WH_kmeans() |
6.8 s | 2.5 s |
WH_adaptive.kmeans() |
7.5 s | 3.1 s |
WH_MAT_DIST(), WH_hclust() |
3.0 s | 1.2 s |
registerMH() |
5.9 s | 2.1 s |
SSQ_RCPP() (C++, used by WH.SSQ(),
WH.var.covar(), WH.correlation()) no longer
builds a dense diagonal matrix of the weights (of size G x G, G being
the number of points of the common grid of cdf values) for each
individual; the sum of the weights is now stored in a double instead of
a float.WH.SSQ() on the already prepared data:
they now use the fast c_ComputeFASTSSQ().WH_kmeans() and WH_adaptive.kmeans(): the
prototypes are computed without copying the columns of the data of each
cluster (new c_ComputeFASTSSQ_sel()).PREPARE_A_VEC_MAT() (C++, puts a set of histograms on a
common grid of cdf values) computes the quantiles with a single ordered
scan of the grid; COMP_Q_VECT() scans the cdf only once for
sorted levels; c_Prepare() no longer copies and transposes
the data.registerMH(), the arithmetic of histograms with numbers
and the regression with simplify = TRUE avoid repeated
validity checks and quantile computations.WH.1d.PCA(): INERTIA (total inertia)
was the sum of the eigenvalues returned by
FactoMineR::PCA(), which since FactoMineR 2.15 are only the
first ones; it is now computed from the percentage of inertia of the
first axis.
HTS.moving.averages(): each average was assigned the
timestamp of the element following the central one of its window;
weights of length different from k are now an
error.
HTS.predict.knn(): with ties in the distances,
which() returned several positions (with a warning) and
could select wrong neighbours; k larger than the available
history is now an error.
WH_2d_Adaptive_Kohonen_maps() with
init.weights = "PROP": the condition
schema == 2 || 4 was always true, so the initial weights of
schemas 1, 3 and 5 were overwritten; for these schemas the vector of
initial weights also had twice the needed length.
d1 - d2 failed (object 'tmp' not found)
when the two distributions share the same cdf, e.g. after
registerMH().
a * d and d * a with
a < 0, and a - d, returned (or failed to
create) objects with a decreasing x; they now return the
distribution of a * X and a - X.
plot() of a MatH (type
"HISTO") cut the densities at the fixed value 2: variables
with a narrow range were drawn as flat, truncated histograms. The
heights are now normalised within each variable.
WH.plot_multiple_indivs() drew the histograms of the
individuals as flat lines (the normalised domain and densities were
computed but not used). The size of the histograms is now set
automatically: strx (default 0.1) is the average width of
the histograms and stry (default 1/8) the height of the
highest density, as fractions of the ranges of the coordinates of the
individuals on the two axes. Each histogram is centred on its mean, and
histograms with very narrow bins (e.g. quantiles with ties) are drawn on
30 equal-width bins to avoid spikes.
The printout of a histogram with many bins used two different formats for the bins.
[ on a MatH object: columns or rows
selected by name got NA names; negative and logical indices
are now allowed (e.g. BLOOD[-1, ],
BLOOD[, "Cholesterol"]).
Five vignettes have been added:
vignette("HistDAWass-01-data"): creating histograms
(distributionH, data2hist()) and matrices of
histograms (MatH), also from a data frame of raw data;vignette("HistDAWass-02-basic-statistics"): statistics
of histograms and of histogram variables (means, quantiles, distances,
covariances, correlations);vignette("HistDAWass-03-clustering"): k-means and fuzzy
c-means, also with adaptive distances, and hierarchical clustering;vignette("HistDAWass-04-pca"): principal component
analysis of one or more histogram variables;vignette("HistDAWass-05-regression"): the two-component
linear regression of histogram variables.c_WH_ADPT_KMEANS_TOTALSSQ() (used by
WH_adaptive.kmeans() and the adaptive Kohonen maps): the
vector of cluster weights assigned to each row of the weight matrix was
allocated with k (number of clusters) elements instead of
vars (number of variables). When k < vars
the remaining variables read uninitialised memory. This made the
examples fail with the forthcoming Rcpp release (Rcpp PR #1513, which
checks the length of vectors assigned into matrix rows). Thanks to the
Rcpp team for the report and the patch.1e-8 is assigned to each empty bin and the cdf is
renormalised. This happens when a distributionH (or
TdistributionH) object is created, in
data2hist() (that used a weight of 0.001 counts), in
checkEmptyBins() and in the internal C++ registration of
distributions (that used 1e-14). Up to 1.0.8 a distribution
with an empty bin produced misaligned quantiles when several
distributions were put on a common grid of cdf values, with errors in
the computed distances (up to 7% in the China_Seas
data).China_Seas, China_Month and
Age_Pyramids_2014 datasets contained 683 distributions with
empty bins: they have been fixed accordingly.compQ() returned wrong quantiles for distributions with
repeated values in x
(e.g. compQ(distributionH(c(1,2,2,3), c(0,.3,.6,1)), 0.7)
returned 2 instead of 2.25).compP() returned wrong values for distributions with
empty bins.distributionH objects never
checked that p is not decreasing;
distributionH(x) with a single value now creates an impulse
distribution as documented.whichLowerEQThanX() (C++) could read one element beyond
the end of the vector; c_DISTA_M2() (C++) accessed the
second element of a list of length one when data had a single
variable.distributionH,
TdistributionH, MatH, HTS) and
plotPredVsObs() now draw the plot on the current graphic
device and return the ggplot object invisibly. Up to 1.0.8
the distributionH and MatH methods returned
the object without drawing it, so nothing was shown inside loops,
functions or source().WH.1d.PCA() no longer opens new graphic devices with
dev.new(): the four plots of each factorial plane are drawn
on the current device (2 x 2 panels), restoring the graphical parameters
on exit. In the plot coloured by the standard deviations the densities
were centred on the standard deviation instead of the mean.WH.MultiplePCA() gains a graph argument
(default TRUE) passed to
FactoMineR::MFA().WH.plot_multiple_Spanish.funs() and
WH.plot_multiple_indivs() no longer print their plots
twice; WH.plot_multiple_indivs() failed with
stat different from "mean", and its cut of the
highest densities was applied to the wrong column.VIOLIN plot of HTS objects showed the
data of the first block of timestamps in all the rows of the plot:
fixed. HTS plots are now a single ggplot with
one facet for each block of maxno.perplot timestamps.plotPredVsObs() returns a single ggplot
(faceted by object) instead of drawing several plots in a grid of
viewports.linewidth instead of size for
lines, guides(fill = "none"), aes() with the
.data pronoun instead of aes_all() and
with().rbind() in loops): summaryHTS() and the
HTS violin plot are about twice as fast.data2hist(algo = "base") no longer draws a histogram as
a side effect.browser() calls from the code of the
adaptive Kohonen maps and of the adaptive weights.Config/roxygen2/version replaces
RoxygenNote): descriptive titles, a single usage for each
method (no duplicated generic/method usage, no initialize
methods in the usage of the classes), accurate return values, documented
slots of the classes, dataset formats and examples, and an overview of
the package in ?HistDAWass.?"Arith-distributionH"),
WH.SSQ/WH.var.covar/WH.correlation
and their two-matrix versions,
WH.vec.sum/WH.vec.mean, WH.bind*,
registerMH/is.registeredMH,
get.cell.MatH/set.cell.MatH and the
get.MatH.* accessors.\dontrun{} examples replaced by
\donttest{} (they all run).checkEmptyBins() has a method for MatH
objects, useful for objects created with previous versions of the
package.c_MM_L2_SQ_WASS_D(),
c_DISTA_M(), c_Fast_D_Mat(),
c_STEP_2_ADA_KMEANS(),
c_ComputeFast_L2_SQ_WASS_DMAT()) no longer allocate
temporary vectors nor access S4 slots in the inner loops, and
c_WH_ADPT_KMEANS_TOTALSSQ() computes cluster-invariant
distances once. Results are numerically identical to 1.0.8; on a 300 x 4
matrix of histograms WH_MAT_DIST() and
WH_hclust() are about 20 times faster,
WH_kmeans() about 6 times,
WH_adaptive.kmeans() about 4 times and the Kohonen maps
about 3 times faster.Minor bug fixes fo compatibility isseues with ggridges.
Added the WH_WASS_MAT_DIST for computing a distance matrix
Fixed some minor bugs. Added the names of columns and the rows in weights matrices of adaptive distances based methods.
Fixed some minor bugs.
Fixed some visualizations accordingly to the new ggplot2 release.
#HistDAWass 1.0.2 What’s new ##(Histogram-valued Data analysis using Wasserstein metric)
Soelf Organizing maps have been improved for speed and aptions.
#HistDAWass 1.0.1 What’s new ##(Histogram-valued Data analysis using Wasserstein metric)
Some bugs have been resolved when processing data with a single variable.
#HistDAWass 1.0.0 What’s new ##(Histogram-valued Data analysis using Wasserstein metric)
Now, a set of methods for MatH (WH.SSQ, WH.vec.mean, and all the methods using them) are much faster due to RCPP implementation.
#HistDAWass 0.1.7 What’s new ##(Histogram-valued Data analysis using Wasserstein metric)
Some general improvements have been added for reducing elaboration times.
unsupervised learning
WH_kmeans now returns more informative outputs
principal components analysis
Multiple factor analysis has new functions for plotting results
#HistDAWass 0.1.6 What’s new ##(Histogram-valued Data analysis using Wasserstein metric)
unsupervised learning
fuzzy c-means and kohonen maps are 10 times faster
#HistDAWass 0.1.5 What’s new ##(Histogram-valued Data analysis using Wasserstein metric)
some minor bug fixed
#HistDAWass 0.1.4 What’s new ##(Histogram-valued Data analysis using Wasserstein metric)
some minor bug fixed
#HistDAWass 0.1.3 What’s new Some fixes to the worksapce
#HistDAWass 0.1.2 What’s new ##(Histogram-valued Data analysis using Wasserstein metric) > From raw data to histograms
A new function was added for generating a histogram from a set of raw data
Visualization
Now for plotting, ggplot2 package is used generally.
A new plot for visualizing Histogram Time Series was added.
A new plot for comparing obversed vs predicted histogram was added.
A new plot for showing the differences between an observed and a predicted histogram was added.
Regression and predictive techniques
A new function for computing Goodness of Fit of a model was added