* CHANGES IN dplR VERSION 1.8.0 File: detrend.R, detrend.series.R, i.detrend.series.R, man/detrend.Rd, man/detrend.series.Rd, tests/testthat/test-rwi.R ----------------------------------------------------------------- - detrend() and detrend.series() now default to method = "Spline". The default was every method, so detrend(rwl) with no method returned a list with one data.frame per series and a column per method, not a set of indices: chron() and every other function that takes indices could not use it. It had been that way since at least dplR 1.0, and almost nobody saw it because every example and the workshop material name a method. detrend.series(y) likewise returned a data.frame of every fit and now returns one vector. Code that relied on the old default must now name the methods. i.detrend.series() does, so it still offers every method. File: as.rwi.R (new), summary.rwi.R (new), rwi.image.R (new), window.rwl.R (new), Extract.rwl.R, spag.plot.R, common.interval.R, man/common.interval.Rd, tests/testthat/test-common.interval.R (new), interseries.cor.R, corr.rwl.seg.R, helpers.R, tests/testthat/test-empty-series.R (new, empty and short series), detrend.R, rcs.R, cms.R, i.detrend.R, i.detrend.series.R, NAMESPACE, man/as.rwi.Rd (new), man/window.rwl.Rd (new), man/spag.plot.Rd, man/detrend.Rd, man/rcs.Rd, man/cms.Rd, man/i.detrend.Rd, man/i.detrend.series.Rd, vignettes/intro-dplR.Rmd, tests/testthat/test-rwi.R (new) ----------------------------------------------------------------- - New class "rwi" for ring-width indices. detrend() (one method), rcs() and cms() now return class c("rwi", "data.frame"). Before, detrend() returned a plain data.frame, and rcs() and cms() returned class "rwl", because they wrote the indices into a copy of the ring widths: every check in dplR then took their indices for widths. No function could tell indices from widths, so none could warn when given the wrong one; rwi.stats() on raw ca533 widths gives rbar.eff 0.350 against 0.423 on the indices, with no warning. The class does not inherit from "rwl", so that the two can be told apart. i.detrend() also returned class "rwl", for the same reason, and now returns class "rwi". - An rwi object carries attr(x, "dplR.detrend"), saying which function and method made it, the settings, and whether the indices are differences (centred on 0) or ratios (centred on 1). It also carries the provenance record of the widths it was made from. - i.detrend() records the method chosen for each series, named by series, in attr(x, "dplR.detrend")$method; before, the choice was made at the keyboard and not kept anywhere. i.detrend.series() attaches its choice to the result as attr(x, "method"). - New as.rwi(), and methods for rwi: `[` and subset() (the rwl methods, which now also keep the detrending record, and name the class in their warning), time(), time<-, summary() and plot(). - summary() on rwi describes the indices as a collection: how they were made, span and common interval, rbar.eff, EPS and SNR from rwi.stats(), and per series its span, mean, sd, ar1 and correlation with the mean of the others from interseries.cor(). The print lists only the series that do not correlate with the others at 'pcrit' (default 0.05); as.data.frame() gives the full per-series table. It takes 'ids' to group cores by tree, as rwi.stats() does, and says which way rbar and EPS were counted: without ids, each series is its own tree. On ca533, grouping 34 cores into 21 trees moves rbar.eff from 0.423 to 0.475. It ends by pointing to summary(x, ids = ) (when ids were not given), rwi.stats.running() and corr.rwl.seg() for what the summary cannot show. On co021 detrended by "Mean", with one series shuffled, that series is the one listed (r = 0.02, p = 0.29). - plot() on rwi gains plot.type = "image": every index as a coloured cell, years across and series down, brown below 1 (or 0 for differences) and green above, each side clipped at its 0.99 quantile. It shows at a glance a growth trend the detrending left in: detrend(co021, method = "Mean") leaves the juvenile trend as a green run at the start of nearly every series. The default is still the spaghetti plot. - The intro vignette uses summary() on the indices in place of rwi.stats(), and shows the image plot for the Spline indices and for "Mean", as a check on the detrending. - spag.plot() draws rwi objects around 1 (or 0 for differences) instead of around each series' mean, so the grey line under each series is the value it should sit at. Widths are drawn as before. It takes an rwi object without the "not class rwl" warning. - New window() methods for rwl, rwi and crn: window(x, 1800, 1900) takes those years, keeping class and records. A window that runs past the data warns and says which years came back; one that misses the data is an error. The `[.rwl` warning about broken years now suggests window(). Series with no values in the window are dropped, with a message naming them: window(ca533.rwi, 1800, 1899) would otherwise hold four columns of NA (CAM132, CAM152, CAM161, CAM201 end before 1800), on which summary(), interseries.cor(), corr.rwl.seg() and detrend() fail with messages that do not say why. - subset() on rwl and rwi drops series left with no values, with the same message; subset(ca533, time(ca533) > 1900) held the same four empty columns. `[` still keeps them, so x[i, j] always returns the columns j names and anything lined up with them stays lined up. - interseries.cor() and corr.rwl.seg() drop series with no values, with a message naming them. One such series stopped either function with "'ts' object must have one or more observations"; the numbers for the other series are the same as with it removed by hand. - interseries.cor() and corr.rwl.seg() give a series too short to correlate an NA correlation, with a message naming it, instead of stopping. A series with fewer than 3 years in common with its master (after prewhitening, which drops the first few values of each series) stopped either function with cor.test()'s "not enough finite observations", and a series with 1 value stopped them in ar() with "'order.max' must be >= 1"; neither named the series, and every other series' result was lost. It happens at the ends of year windows: 4-23% of 50- and 100-year windows of the bundled collections have a series with 1-3 values. Over every 50- and 100-year window of ca533, co021, gp.rwl, wa082, nm046 and anos1, 130 of 2204 calls failed before and none now. With Spearman or Kendall, 2 years in common used to give a correlation of +/-1; it is now NA. Other results are unchanged. - ar.func(), used for prewhitening throughout, returns a series with fewer than 2 values unchanged (the order-0 model) instead of stopping in ar(). detrend.series(method = "Ar") on a one-value series now records order 0 instead of stopping. - summary() on rwi lists series with no values at all (x[rows, ] can still make them) and leaves them out of the statistics, instead of failing. - common.interval() on rwi returned class "rwl", with the "not class rwl" warning; it now returns indices as class "rwi", without the warning. The result had no NA and no empty columns on every bundled data set, all three types, indices, an interior gap and empty input columns. - common.interval() on one series, or on series that never overlap, returned a 0 x 0 object with no warning, which a PCA or correlation then fails on with a message about dimensions. One series (after leaving out series with no values) now returns its measured years, its own common interval. Two or more series with no year in which any two overlap, or no series with values, is an error that says so. - Values are unchanged. With several methods, detrend() still returns a list of data.frames, one per series, which are not classed; nor are the fitted curves from return.info = TRUE. - Functions now check which kind of series they are given, by class. Each warning names the function, says what goes wrong, and says how to relabel the data if the class is what is wrong (indices read with read.rwl() come back as class "rwl"). They are warnings, and the function goes on, as before. - Wanting widths, and warning on class "rwi": detrend(), rcs(), cms(), i.detrend(), bai.in(), bai.out(), pointer(), strip.rwl(), ssf() and rwl.report(). These used to give the generic "not class rwl" warning; rwl.report() stopped. - Wanting indices, and warning on class "rwl": chron(), chron.ars(), chron.stabilized(), rwi.stats(), rwi.stats.running() and sss(). These gave no warning before; rwi.stats(ca533) gives rbar.eff 0.350 against 0.423 from the Spline indices. A plain data.frame or matrix is still taken without a word. sss() warns once, not again from the rwi.stats() inside it. - Taking either class quietly: interseries.cor(), corr.rwl.seg(), corr.series.seg(), ccf.series.rwl(), series.rwl.plot(), seg.plot(), spag.plot(), xdate.floater(), sgc() and rwl.stats(). These gave the "not class rwl" warning on rwi, and returned what they returned as widths. Anything else is still coerced to rwl with that warning. The checks are check.rwl(), check.rwi() and check.rwl.rwi() in helpers.R. strip.rwl() detrends its indices a second time on purpose and relabels them first, so it does not warn itself. File: detrend.R, detrend.series.R, man/detrend.Rd, tests/testthat/test-empty-series.R ----------------------------------------------------------------- - detrend() drops a series with no values, with a message naming it, as window() and interseries.cor() do. One such series used to stop the whole call with "all values are 'NA'", which from the parallel loop came out as "task 23 failed" and named nothing. The other series' indices are the same as with it removed by hand. 'y.name' is cut down with the series, and must now have one name per series. - detrend.series() names the series (its 'y.name') when it stops on all-NA values or on NA inside the series, and the second message points to fill.internal.NA(). An interior gap given to detrend() now says which series has it, though still inside "task N failed". File: as.bai.R (new), bai.in.R, bai.out.R, helpers.R, detrend.R, rwl.report.R, common.interval.R, Extract.rwl.R, window.rwl.R, NAMESPACE, man/as.bai.Rd (new), man/bai.in.Rd, man/bai.out.Rd, man/check.rwl.Rd, man/window.rwl.Rd, man/as.rwi.Rd, tests/testthat/test-bai.R (new) ----------------------------------------------------------------- - New class "bai" for basal area increment. bai.in() and bai.out() now return class c("bai", "data.frame"); they returned class "rwl", because they wrote the areas into a copy of the widths. Once functions began to check which kind of series they were given, that label made chron(bai.in(x)) -- a BAI chronology, an ordinary thing to build -- warn that chron() had been given ring widths, as the bai.in() and bai.out() examples showed on R-hub. The areas are unchanged. - A bai object carries attr(x, "dplR.bai"), saying which function made it and whether 'd2pith' or 'diam' was given or estimated from the widths, and the provenance record of the widths. - New as.bai(), and methods for bai: `[`, subset(), time(), time<-, window(), summary() (rwl.stats()) and plot() (seg.plot() or spag.plot()). common.interval() keeps the class. - Which functions take it: chron(), rwi.stats() and the other functions that want indices take it quietly, as they take a plain data.frame; so do the crossdating functions, the plots, common.interval(), and detrend(), since detrending BAI is established practice. Functions that want ring widths -- bai.in(), bai.out(), rcs(), cms(), rwl.report() and the rest -- warn, and treat the areas as widths. check.rwl() gains 'bai.ok' (default FALSE) for detrend() to use. File: rcs.R, cms.R, i.detrend.R, man/rcs.Rd, man/cms.Rd, man/i.detrend.Rd, tests/testthat/test-empty-series.R ----------------------------------------------------------------- - rcs(), cms() and i.detrend() drop a series with no values, with a message naming it, as detrend() now does. rcs() used to stop with "NA/NaN argument"; cms() returned the empty series as a column of NA in the indices; i.detrend() stopped when the series' turn came, and every method already chosen at the keyboard was lost. The other series' indices are the same as with it removed by hand. cms() still requires a 'po' row for every series, empty or not. i.detrend() cuts 'y.name' down with the series, and it must now have one name per series. File: rwl.check.R, tests/testthat/test-rwl.check.R ----------------------------------------------------------------- - New check RWL_EMPTY_SERIES (warning) for a series with no values at all. Empty series used to be reported only as copies of each other (RWL_DUP_SERIES, "one core may be archived twice"), because they hash alike; a lone empty series was not reported at all. They are now left out of the duplicate check. A file cannot hold one, but an object can: ca533[time(ca533) %in% 1800:1899, ] holds four. File: vignettes/intro-dplR.Rmd (new), man/ssf.Rd, README.md, zzz.R, xdate.report.R, man/xdate.report.Rd, tests/testthat/test-xdate.report.R ----------------------------------------------------------------- - xdate.report() gains a 'flagged' element: a data frame with one row per flagged segment (series, segment years, flag, correlation as dated, best lag, correlation at that lag, gain, and whether a B is weak at every lag). It is the report's flagged segments table as data; before, getting it meant indexing the series-by-segment flags matrix. It has no rows when nothing is flagged, and is NULL when nothing was crossdated, as 'flags' is. - New vignette, "Getting Started with dplR": reading, describing, detrending, building a chronology and checking crossdating on co021, using dplR and base R only. It points to Learning to Love dplR for more. dplR has had no introductory vignette since the Rnw vignettes were removed in 2022. - README.md and the startup message point new users to the vignette as well as the book. - ssf.Rd called the workshop book "Learning to Love R", and "decried" is now "described". File: normalize1.R, normalize.xdate.R, helpers.R, rwi.stats.running.R, detrend.series.R, man/rwi.stats.running.Rd, man/corr.rwl.seg.Rd, man/detrend.series.Rd, man/detrend.Rd, tests/testthat/test-difference.R (new) ----------------------------------------------------------------- - Data that can be negative (GitHub issue 22, from Stefan Klesse). rwi.stats() and rwi.stats.running() gave wrong rbar, EPS and SNR, with no warning, for series with a negative mean, as indices from detrend(difference = TRUE) often have. normalize1() divided each series by its mean, which flips a series with a negative mean and every correlation with it. On ca533 detrended with difference = TRUE, subtracting 3 from one series moved rbar.tot from 0.416 to 0.373. The grand-mean check added for this only caught data whose mean over all series was negative, and asked users to pass rwi + 1. normalize.xdate(), used by interseries.cor(), corr.series.seg(), ccf.series.rwl() and the other crossdating functions, did the same: on that data, series 1's interseries.cor went from 0.541 to -0.541. Both now subtract the mean from data with negative values rather than divide by it. That keeps what the division was for, putting every series on the same level so each counts equally in a master or a tree mean (a biweight mean in particular is not shift- invariant), without flipping signs. Positive data are divided by the mean as before, so results for ring widths and ratio indices are unchanged. The 'n' and 'nyrs' filters divide by a smooth curve, so they now stop on data with negative values rather than return meaningless ratios. The grand-mean check is gone. - detrend.series() with difference = TRUE no longer requires the fitted curves to be positive. Positivity is only needed to divide. Before, a Spline, AgeDepSpline or Friedman fit that went below zero was replaced by the mean, an unconstrained ModNegExp or ModHugershoff fit that ended below zero fell back to a line or the mean, and "Ar" set negative values to zero, which turned an all-negative isotope series into a constant. With difference = TRUE these fits are now used as they are, and zeros in the data are no longer recoded to 0.001. constrain.nls = "always" and "when.fail" keep their documented bounds (k >= 0, d >= 0) either way. Division (difference = FALSE) is unchanged. File: man/skel.plot.Rd, man/latexify.Rd ----------------------------------------------------------------- - Two reference URLs failed CRAN's URL check. skel.plot.Rd cited Meko's Tree-Ring MATLAB Toolbox with the MATLAB Central home page, which answers the checker with 403; it now links the toolbox page itself, https://dmeko.ltrr.arizona.edu/toolbox.html. latexify.Rd linked Tralics over https, where INRIA's certificate has expired; it now uses http, which serves the same page. File: xdate.report.R (new), man/xdate.report.Rd (new), NAMESPACE, tests/testthat/test-xdate.report.R (new) ----------------------------------------------------------------- - New function xdate.report(), with format() and print() methods: a COFECHA-style crossdating report in the layout of the "COFECHA output:" block of the ITRDB correlation-stats files. Chris Guiterman and Ed Gille asked for a way to regenerate that block with dplR. It has the summary header, the correlation of series by segments (with A and B flags), a list of the flagged segments with the lag and the gain in correlation, the descriptive statistics, the errors and warnings from rwl.check(), and notes on how it was made: "Report generated using dplR (R )", the time, the MD5 of the measurement file when given a path, the filtering, the master, the correlation and critical value, the segments, and every setting that had to change to fit the data. write.xdate.report() saves it as fixed-width text, as Markdown or as HTML, chosen from the file name (.md, .html) or by 'type', and format(x, type = ) gives the lines. The HTML is one self-contained page in base R, with no new dependency: plain tables, A and B cells shaded, B flags that are weak at every lag greyed, text from the data escaped, and a small inline stylesheet that also prints. The COFECHA "PART 5:" and "PART 7:" labels are left out. The summary and totals averages are weighted by the number of years in each series. That is what COFECHA does: on cana295 the plain mean of its standard deviation column is 0.743, its header says 0.760, and the weighted mean is 0.760, and the same holds for the intercorrelation (0.524) and the autocorrelation (0.895). It is built on corr.rwl.seg() with lag.max, nyrs = 32, ar.order.max = 3, pcrit = 0.01 and Pearson correlation, which is COFECHA's, so an A flag is exactly a correlation under the critical value the report prints (0.3281 for 50-year segments). It began as a script in the ITRDB clone, and three things changed on the way in: the B flags now require the whole shifted window (the script kept any lag with half its pairs, so it raised false flags at the ends of the record), the 32-year spline its notes described is now applied (it never passed nyrs), and the filtered block of the descriptive statistics is labelled "dplR filtered", because it is not COFECHA's filtered series. ar.order.max = 3 is lower than corr.rwl.seg()'s default. After the 32-year spline, AIC picks an AR order near 20 (17 to 23 on co021), and prewhitening loses that many years, and the segments with them, at the start of every series; COFECHA's AR column reads 1 or 2. On co021 the cap tests 718 segments instead of 690. A B flag whose best-lag correlation is still under the critical value is marked "weak at every lag" in the flagged-segment list, and counted in the summary. On wa082 all four B flags are like this (lags of -9, -9, -8 and +8, best r 0.24 to 0.31), and the ITRDB technician read COFECHA's flags there the same way in 1996: "ALL FLAGS = LOW CORRELATIONS WITH MASTER". On co021 with the planted missing ring, none of the nine B flags is weak. The letter stays B. It differs from COFECHA in ways the report states: only complete segments are tested, where COFECHA also tests partial segments at the ends of a series. A series whose spline cannot be fitted is described but left out of the crossdating, and named with the reason, rather than stopping the report; a record too short for 50-year segments gets shorter ones; and a site with fewer than three usable series gets the descriptive statistics alone. File: insert.ring.R, tests/testthat/test-insert.ring.R (new), man/corr.rwl.seg.Rd, man/ccf.series.rwl.Rd, man/corr.series.seg.Rd, man/xskel.plot.Rd, man/xskel.ccf.plot.Rd, tests/testthat/test-corr.rwl.seg.R ----------------------------------------------------------------- - delete.ring() with fix.length = TRUE, the default, gave the NA it pads on an empty name, so the years read "", 2002, ... instead of 2001, 2002, .... Anything taking years from the names got an NA year, and insert.ring() refused the result with "input data must have consecutive years", so a missing ring and a false ring could not be chained. With fix.length the output is now named with the input's years. The values are unchanged. insert.ring() did not have the problem. New tests cover both functions, which had none. - The crossdating examples now follow one planted fault: the 1500 ring deleted from co021 series 641143 with delete.ring(). corr.rwl.seg() finds it (-1 in every bin before 1500) and also shows it paired with a false ring from insert.ring(), ccf.series.rwl() and corr.series.seg() confirm it, and xskel.plot() and xskel.ccf.plot() show a window after it and one across it. ccf.series.rwl() and corr.series.seg() already deleted ring 325, which is 1500, by index; their results are the same. xskel.plot() and xskel.ccf.plot() deleted 1825 instead, and passed the rwl with the original 641143 still in it, so the master held a correctly dated copy of the series being tested. They now drop it, as the other examples do. File: rwl.check.R, tests/testthat/test-rwl.check.R, man/rwl.check.Rd, man/xskel.ccf.plot.Rd ----------------------------------------------------------------- - The lag in rwl.check()'s RWL_DATING_LAG finding had the opposite sign to ccf.series.rwl() and corr.rwl.seg(): a series missing a ring was reported at +1 here and -1 there. It called ccf(series, master); it now calls ccf(master, series), so a negative lag means missing rings in the series throughout dplR. Only the sign of 'value' changes. Which series are flagged, and the correlations in the message, are the same, because the best correlation at lag k one way round is the best at -k the other. The test compared abs(value), which is why this was not caught; it now checks the sign in both directions and against corr.rwl.seg(). rwl.check() is new in this version, so no released output changes. The message now says which way the error most likely runs: a missing ring for a negative lag, a false ring or a ring measured twice for a positive one. - man/xskel.ccf.plot.Rd named the argument that sets the COFECHA convention 'switch.x'. It is 'series.x'. File: corr.rwl.seg.R, plot.crs.R, tests/testthat/test-corr.rwl.seg.R, man/corr.rwl.seg.Rd, man/plot.crs.Rd ----------------------------------------------------------------- - New argument 'lag.max' in corr.rwl.seg(), default 0. Each segment is also correlated against its leave-one-out master shifted by every lag from -lag.max to lag.max, and the best position is returned in two new matrices, 'best.lag' and 'best.rho', with the shape and names of 'spearman.rho'. A third new element, 'lag.max', records the argument. No existing element changes, and 'flags' is still the p-value test alone. At lag.max = 0 no extra correlations are computed, 'best.lag' is 0, 'best.rho' is a copy of 'spearman.rho', and the plot is the same as before. Until now nothing in corr.rwl.seg() shifted a segment, so a segment that correlated 0.31 as dated and 0.62 at lag -1 was significant where it sat and was not flagged at all. COFECHA separates two cases: A, below the critical value but best where it is dated (weak, not misdated), and B, better at some other position (a dating hypothesis). Over 36 ITRDB sites and 5,301 segments there were 197 A against 395 B. The letters are not stored: B is best.lag != 0 and A is best.lag == 0 & p.val >= pcrit, which the Rd example builds. plot.crs() draws B segments in purple when lag.max > 0. The sign matches ccf.series.rwl() with series.x = FALSE: negative lags mean missing rings in the series. Each year of the segment is paired with the master value k years away, and a lag whose window is not complete in the master is not tested, the same rule as at lag 0. So segments within lag.max years of either end of the record are only searched one way, and the Rd says that floaters sit exactly where the test is blind. It also says that a B is a hypothesis: a one-year shift for a 0.05 gain is often noise, which is why the gain is returned and not just the lag. The Rd explains how to read the lags along a series: a missing ring carries -1 into every bin before it, so the error is where the lag changes, and a run of -1 with correct dating on both sides is a ring in the wrong place (missing at the later end, extra at the earlier end), which a whole-series test such as rwl.check()'s RWL_DATING_LAG does not see. The example plants both faults in co021 series 641143 and shows each pattern, and the test checks the second on nm046, which is smaller and faster. File: normalize1.R, normalize.xdate.R, helpers.R, corr.rwl.seg.R, corr.series.seg.R, ccf.series.rwl.R, interseries.cor.R, rwi.stats.running.R, series.rwl.plot.R, xskel.plot.R, xskel.ccf.plot.R, xdate.floater.R, tests/testthat/test-normalize.R, man/corr.rwl.seg.Rd, man/corr.series.seg.Rd, man/ccf.series.rwl.Rd, man/interseries.cor.Rd, man/rwi.stats.running.Rd, man/series.rwl.plot.Rd, man/xskel.plot.Rd, man/xskel.ccf.plot.Rd, man/xdate.floater.Rd ----------------------------------------------------------------- - New arguments 'nyrs' and 'ar.order.max' for every function that normalizes series before correlating them: corr.rwl.seg(), corr.series.seg(), ccf.series.rwl(), interseries.cor(), rwi.stats.running(), series.rwl.plot(), xskel.plot(), xskel.ccf.plot() and xdate.floater(). Both default to NULL, which gives the same results as before. They sit with the arguments they belong with, as n, nyrs, prewhiten, ar.order.max (prewhiten, n, nyrs, ar.order.max in rwi.stats.running()), so a call that passed prewhiten or later arguments by position now shifts. Every such shift lands a logical or character value in 'nyrs' or 'ar.order.max', and both refuse it with an error, so no old call runs with a different meaning. Of the 12 CRAN packages that depend on or suggest dplR (checked by searching their sources, not by revdepcheck), only measuRing calls any of these functions: crossRings() wraps corr.rwl.seg() and ccf.series.rwl(), and it passes arguments by name through do.call(), so the reordering does not affect it. 'nyrs' divides each series by a smoothing spline (caps()) before prewhitening, as COFECHA does with a 32-year spline. The indices are the ones detrend(method = "Spline", nyrs = nyrs) gives, so corr.rwl.seg(rwl, nyrs = 32) is the same as detrending first and passing the indices. It cannot be combined with the Hanning filter 'n'. Unlike detrend(), a spline that is not all positive stops with the series named, rather than falling back to the mean for that one series, and so does a series with internal NA. 'ar.order.max' limits the order of the AR model used to prewhiten. The request behind 'nyrs' was to lose fewer years at the start of each series, and the spline does not do that by itself: the years lost at the start equal the AR order, and AIC picks a high order for a spline-filtered series. On co021, ca533, gp.rwl and wa082 the median years lost with nyrs = 32 is 10 to 25.5, against 4 to 15 with the old default. With nyrs = 32 and ar.order.max = 3 it is 1 to 3. corr.rwl.seg(), corr.series.seg() and xskel.plot() each had a local variable called 'nyrs' holding the number of years. In corr.rwl.seg() and xskel.plot() it was assigned before the normalizing step and would have overwritten the new argument: every call would have been divided by a spline with the number of years as its rigidity, whether or not 'nyrs' was set, and a call with 'n' set failed with 'n' and 'nyrs' both set. This was caught during development and never released. The local is renamed 'n.yrs' in corr.series.seg() and xskel.plot(), and removed from corr.rwl.seg(), where it was not used. test-normalize.R checks that the arguments arrive intact. The corr.rwl.seg() example that detrended with a spline passed 'plot = FALSE', which is not an argument, so it plotted anyway. It now uses 'nyrs' and 'make.plot = FALSE'. File: read.tucson.R, rwl.check.R, tests/testthat/test-read.tucson.R ----------------------------------------------------------------- - read.tucson() no longer drops a series in silence. Every refusal in the reader works one cell at a time: a value that cannot be read becomes NA, and the NA rows are then filtered out before the data is cast to wide. A series whose every cell was refused loses every row it had, so no column is ever created for it. The file names the series, the reader read its lines and refused each of them for a stated reason, and the caller got an object with one fewer series in it and nothing anywhere saying which one left. can697 is the case: 94 series in the file, 92 in the returned object, 27 warnings about decade lines, and not one word about B22B1 or B22B1b. Which two had gone missing was something the caller had to work out by counting columns against the file. can712 is the same shape, 74 series returned of 78. The per-line reasons were already reported. What is added is the consequence, which is the part that cannot be reconstructed from the returned object: a SERIES_DROPPED event naming the series and how many lines each had, and saying that they are not in what you were handed. rwl.check() reports it as RWL_SERIES_DROPPED. Worth recording how this survived a test suite that covers the reader closely. The test for a line the reader cannot read asserted all(is.na(r[["RH17A"]])) -- and RH17A is not in the object at all, so that is all(is.na(NULL)), which is TRUE. The assertion passed on the absence it was meant to rule out, and read as documentation it said the silent drop was intended behaviour. It now asserts that the series is absent and that the drop is reported. Measured over the ITRDB clone, 10,162 files: two files were losing a series in silence, can697 and can712, six series between them. Both are in the 20 that would newly fail if COLUMN_LAYOUT and PAST_COL72 became fatal, so if that change lands this check has nothing left to fire on archive-wide. It keeps its place for the NON_NUMERIC and NO_MEASUREMENT paths and for files outside the archive, but it is not evidence for the severity change. - read.tucson() reported a measurement split across column 72 on lines where nothing was split. The check exists for a measurement cut in half by the column boundary: a digit in column 72 and another in column 73, so that truncating at 72 eats a digit. mar047's TIZ19A 1900 line turns 116 into 11 that way, 1.16 mm reported as 0.11 mm. The test for the column-72 side was made on a right-trimmed string, so it asked whether the last non-blank character ANYWHERE in columns 1-72 was a digit. Data lines end in a measurement, so that is nearly always true, and the predicate collapsed to "there is a digit at column 73". A trailing per-line count column beginning at column 73 fired it with column 72 blank and nothing split at all -- which is exactly the disposable overflow the check was written to ignore. Now tested at the character that sits at column 72. Measured over the ITRDB clone, 10,162 files: the false positive occurs on one file, europe/bulg002t-noaa.rwl, whose column 72 is blank, whose content ends at column 68 and which carries a 999 at column 73. That file is one of the NOAA template tables that fail for holding no measurements, so nothing that reads today changes. PAST_COL72 fires on 6 files rather than 7, and the 20 files that would newly fail if it and COLUMN_LAYOUT became fatal are the same 20 as before. - The PAST_COL72 message no longer asserts that a digit was lost. It said a measurement was "split across the boundary, so the truncation at column 72 drops part of a number". That is one of two readings and the line does not say which. Reading all six files in the archive that raise it: in four the line really is misaligned and the truncated value is out of range for its own series -- mex125's NIH14B 1990 truncates 3380 to 338 among neighbours of 2570 to 6940. In the other two, ok049 and ok049l, the ten fields are perfectly aligned, the record is complete at column 72, the file pads every other line to 82 columns with blanks (4,925 of 4,926 lines in ok049), and one stray character sits past the boundary. There the truncated value is the right one, and both readers already return it. The message now gives both readings, says the reader takes the shorter one, shows what is past column 72 so the caller can see it, and points at the neighbouring years as the way to tell. What cannot be told from the file, it no longer claims. This settled a severity question that was open. All four genuine cases fail the column-conformance check on the same line and the two stray-character cases do not, so COLUMN_LAYOUT catches every real straddle in the archive and PAST_COL72 has no independent true positive anywhere in 10,162 files. Making it fatal, as was proposed, would refuse ok049 and ok049l -- 889 series between them -- on the strength of two stray keystrokes, in the two cases where it is the only voice and it is wrong. It stays a warning. - read.tucson()'s refusal of a series terminated at two precisions now says what is wrong with the file. It said "core(s) kok3a have different precision flags", which is a fact about a variable inside the parser. It does not say where in the series the disagreement is, which two precisions are involved, what reading the file anyway would cost, or what to do about it. kyrg014's kok3a is the case and the only one in the ITRDB archive: one ID entered as two records, 1642-1898 ending in -9999 and 1900-2005 ending in 999. Read as one series, one half comes out ten times the size of the other. The refusal stays unconditional, and should: there is no reading to hand back, and either choice produces ring widths that are wrong by a factor of ten while looking perfectly ordinary. What is added is the diagnosis -- the marker, its year, both precisions, the cost of guessing, and the remedy, which is to enter each record under its own series ID. - The series recorded in read.tucson()'s provenance events was the wrong one for every event raised while parsing a decade line. Three events -- COLUMN_LAYOUT, NON_NUMERIC and NO_MEASUREMENT -- are raised from inside a data.table j-expression. data.table evaluates j once per group in one reused environment and writes each group's column value into one reused vector, in place. Storing that vector in the event row stored a reference to it, so every such row ended up holding the last group's series id: can697's COLUMN_LAYOUT event came back filed under the alphabetically last series in the file. The message was right throughout, because paste0() had already copied the characters into a new string. Only the structured field was wrong, which is the field a sweep over many files filters on. The values are now copied before they are stored. - read.tucson() now reads a line whose year is not in columns 9-12, instead of discarding it. va024 is the case. Two of its lines write the id wider than the eight characters the format allows, so the year lands at columns 13-16 and columns 9-12 are blank: 25B 1833 324 386 368 <- every other line 25B 1930 108 97 70 <- these two The year field did not parse, so the lines were discarded with a BAD_YEAR warning. They are the only lines for 25B's 1930s and 12A's 1940s, so the file came back with a ten-year hole in each of those series where it plainly holds data -- twenty measurements. read.tucson.legacy() loses them too, silently. This is not the COLUMN_LAYOUT ambiguity and is not treated like it. There, two readings of one line disagree and nothing says which was meant. Here the fixed-column reading is not a reading at all, since it yields no year, so there is one candidate and it is the one any human makes at a glance. The recovery belongs with the dash and bunched-number idioms rather than with guessing. The predicate is narrow, because the cost of a wrong recovery is a fabricated measurement. The id must start at column 1, which stops a line with no id from having its first measurement read as the year; the second token must be exactly four digits, so BC years, which sit at columns 8-12, are left alone rather than rebuilt onto a layout somebody chose; every later token must be an integer and there must be between one and ten of them; and nothing may be too wide for the grid. swe347's stray header line "swed347 3 Lie" fails three of these at once and is still discarded. The event is YEAR_MISPLACED, RWL_YEAR_MISPLACED in rwl.check(), and the report quotes the line and the reading taken from it. The line is rebuilt into canonical form rather than having its year patched, because the tail parser reads columns 13-72: on a shifted line the measurements are not there either. That also means the recovery has to work from the untruncated line. It did not at first, and va024 showed why within minutes of the code being written: the record is pushed right, so its tenth measurement sits at columns 71-76 and truncation at column 72 had already taken it. Nine values came back where the file holds ten. The overflow past column 72 is now kept until after the head parse for exactly this. Keeping it that long means the duplicate-line check names V1 rather than comparing the whole row -- the same comparison it was already making, since V1 and the overflow are the only columns the table has at that point. One known limit, recorded rather than papered over: a shifted line that also carried a trailing per-line count column would read the count as a measurement, because once the record is off its columns nothing separates the two. No file in the archive has both at once, and the ten-value ceiling rules out the common shape, a full decade plus a count. Measured over the ITRDB clone afterwards, 10,162 files: BAD_YEAR falls from 2 files to 1 -- swe347's stray header line is now the whole of it -- and YEAR_MISPLACED is va024 alone. Every other event family holds exactly the files it held before, no file changed its series count, none changed whether it reads or fails, and no file's warning count moved. - The worked example for SECOND_RECORD, in a comment, no longer exists in the data. The comment cites az621's FR-001 and FR-002 sharing a line because a line break is missing. The archive copy of az621 now holds exactly one line longer than 72 characters and that line is the site header; the file reads clean, with no warning. Whether NOAA re-issued it or the copy the comment was written against differed cannot be told from here, so the comment now says what is true and does not guess at the cause. This matters beyond tidiness. Archive-wide SECOND_RECORD fires on 8 files and all 8 are the europe/*-noaa.rwl template tables, which are refused anyway for holding no measurements. The family therefore has no worked example in a file that reads, which is worth knowing before its severity is settled on the strength of one. File: encoding.R (new), read.rwl.R, read.tucson.R, read.sheet.R, rwl.check.R, NAMESPACE, tests/testthat/test-encoding.R (new), man/read.tucson.Rd, man/read.sheet.Rd ----------------------------------------------------------------- - The readers no longer die on a file that is not valid UTF-8, and read.tucson()'s encoding argument now does something. A single accented byte anywhere in the first 30 lines of a file -- an investigator's name in a header comment, which is the ITRDB norm -- killed all three of the new readers, and killed them badly. read.rwl(fname) died in sniff.rwl(), at trimws(), with "input string 3 is invalid UTF-8", before a reader had even been chosen. read.tucson() died inside data.table with "invalid multibyte string, element 3". read.sheet() died at the BOM strip. None of the three messages named the file, the line or the cause, none of the files contained a single non-ASCII ring width, and read.tucson()'s encoding argument -- the obvious remedy -- was accepted and then warned to be ignored. read.sheet() had no such argument at all. Note what was NOT wrong: nothing returned mojibake or a wrong measurement. It crashed. The defect was the diagnosis, not the safety, so the loudness is kept and the explanation added. Encoding is now resolved in three tiers, in encoding.R, shared by sniff.rwl(), read.tucson() and read.sheet() so that all three say the same thing about the same file: 1. Is it valid UTF-8? A decision, not a guess -- UTF-8 is self-validating, and ASCII passes -- so virtually every file takes this path and nothing is reported. 2. Not UTF-8, and encoding was supplied? Use it, and record an ENCODING_DECLARED note. A declared encoding that does not fit the file is an error, not a transcode into replacement characters. 3. Not UTF-8, nothing supplied? Read as latin1, record an ENCODING_ASSUMED event, and warn with the line number and the decoded text the assumption produced. strict = TRUE makes it an error. read.sheet() gains an encoding argument to match. Both events are in rwl.check.catalogue(), so a sweep over an archive can count them. The fallback guesses rather than detecting, and that was measured rather than assumed. stringi is already an Import, so ICU's charset detector was free to use, and it does not work on this corpus: it is a byte-frequency language model needing a volume of non-ASCII text that a file of ring widths never has. On a rwl that is ASCII apart from one accented name it scored the right answer at 0.16 and ranked UTF-16BE alongside it; on a cp1252 file and on a latin2 file the right answer was not in its top three at all. Only on dense prose (480 non-ASCII bytes of French) did it return a usable 0.76. Wiring a 0.16-confidence guess into a decision behind a confident-looking API is precisely the plausible-looking garbage this package refuses to produce, so the detector is used only to enrich a message, never to decide, and stays quiet below 0.7. It also reports the charset only, not the language: at 0.5 it volunteered "language da" for a German name. latin1 is the fallback because two properties matter more than being right. It is total -- all 255 bytes decode, so it can never fail -- and it is lossless, the round trip being byte-identical, so a wrong guess discards nothing and the file can be re-read with encoding set. The usual alternatives destroy the byte for good: iconv(sub="") turns "M\xfcller" into "Mller" and sub="?" into "M?ller". One structural note. In read.tucson() the validity TEST runs on what fread() already returned, so a UTF-8 or ASCII file pays one vectorised call and takes the old fread(fname) path bit-for-bit unchanged. The conversion, however, re-reads with readLines(), because fread() was given blank.lines.skip = TRUE and an index into its result is not a line number in the file: converting in place reported "line 3" for what was line 5 of a file with two blank lines above it. A message that confidently names the wrong line is worse than one that names none. read.fh() gets the same treatment and an encoding argument. This is the format where it matters most in practice: Heidelberg files are favoured in European dendroarchaeology, their HEADER blocks are full of free text -- site, species, project and operator names -- and that text is typed in German, Czech and Polish on machines that were not writing UTF-8. A single umlaut in a Location field used to take read.fh() out at its first grep() with "input string N is invalid in this locale". read.fh() keeps no provenance record, so the tiers report through plain conditions there: a declared encoding is obeyed silently and an assumed one warns. read.crn() and read.tucson.legacy() already honoured encoding and are unchanged. read.tridas() is deliberately untouched: it parses through libxml2, which reads the encoding from the XML declaration and transcodes to UTF-8 itself. A declared ISO-8859-1 TRiDaS file already reads correctly, and undeclared 8-bit bytes are already rejected by the parser, so a check of our own would only break working files. read.compact() does not have the check. File: read.rwl.R, tests/testthat/test-sniff.rwl.R, man/read.rwl.Rd ----------------------------------------------------------------- - Format detection for read.rwl(format = "auto") is now its own function, sniff.rwl(), and has its own tests. It was untestable where it was. The only way to ask what dplR thought a file was had been to read the file and see which reader's output came back, which conflates a detection bug with a parsing bug and requires a fixture to be a valid file of its type before the question can even be put. None of the fixtures in test-sniff.rwl.R is a valid file of anything. Pulling it out found a bug. The test for a csv fired on ANY comma in lines 2-21. It skipped line 1 so that a Tucson header carrying a comma would not be read as a csv (issue #16), but that fixed only the header case: a Tucson file with a comma anywhere BELOW line 1 -- a note in a series field -- was still detected as a csv and handed to read.sheet(), which then refused it. Detection now asks whether every line splits into the same number of fields, which a stray comma in prose does not, so the Tucson cases are excluded on better grounds. The old and new rules were compared directly on that file to confirm it rather than reasoned about. The sheet test also reuses read.sheet()'s own separator detector rather than carrying a second, worse implementation of the same question. A NOAA/NCEI template is now named by the detector, so read.rwl() says what the file is instead of handing it to read.tucson() to fail on. auto still looks for commas only: a tab separated file may be a NOAA template whose header went unrecognised, and that is not a guess for a dispatcher to make. format = "sheet" with a sep argument is one argument away. sniff.rwl() returns the evidence for its answer as well as the answer. When detection picks the wrong reader the useful question is what it saw, and nothing could answer that before. File: read.rwl.R, write.rwl.R, man/read.rwl.Rd, man/write.rwl.Rd, tests/testthat/test-write.sheet.R ----------------------------------------------------------------- - read.rwl() and write.rwl() take format = "sheet", which is the name of the thing: the reader is read.sheet() and the format value now says so, as every other one does. "csv" stays as a synonym. It has been a valid value of read.rwl()'s format since long before read.sheet() existed and cannot be taken away from anyone. It is also the less accurate of the two now: read.sheet() reads tab, semicolon and pipe separated files, so format = "csv" with sep = "\t" is a perfectly ordinary call and reads oddly. New code should say "sheet". The detection line in format = "auto" now reads "Detected a comma separated sheet", which is what it detected -- auto still looks for commas only, deliberately, since a tab separated file may be a NOAA template and that guess is not one to make in a dispatcher. File: read.sheet.R, write.sheet.R, man/read.sheet.Rd, man/write.sheet.Rd, tests/testthat/test-read.sheet.R, tests/testthat/test-write.sheet.R ----------------------------------------------------------------- - read.sheet() and write.sheet() handle comma, tab, semicolon and pipe separators and either decimal mark. Each of the four round trips ca533 identically, both with the separator given and with it detected. A semicolon separated file with decimal commas -- the European Excel default, and a recurring source of confusion for those users -- now reads without any argument at all. The separator is detected by consistency rather than frequency. A comma count is easy to fool, since a species note or a header line holds commas and no structure, whereas a real separator gives every line the same number of fields; among the candidates that manage that, the one carving the file into the most fields wins, which is what keeps a semicolon file with decimal commas from being read as twice as many comma-separated columns. Two things the counting has to get right, both found by tests rather than by reasoning. It counts the separator instead of splitting on it, because strsplit() discards trailing empty fields: a sheet where one series ends before the others -- which is most collections -- then looks inconsistent line to line and no separator is found at all. And it blanks quoted spans before counting, because a series id may legally contain the separator, in which case the header line carries one more comma than every data row. The detected separator and decimal mark are recorded in the provenance record, as SEP_GUESS and DEC_COMMA, but they do not warn and strict does not escalate them. They are decisions the reader made, not defects in the file, and sep = NULL is the default -- so warning would put a warning on every ordinary call. - The NOAA guard is now independent of comment.char. It shipped in place before tab support for the reason that matters here: a NOAA template file is a commented metadata block above a tab separated data table, and as of this release that table parses perfectly well. Reading one would return the ring widths and silently discard the coordinates, species, investigators and DOI above them. The guard scans the file's leading lines rather than the parsed comment block, so someone reading with comment.char = "" cannot slip past it, and it is tested against a tab separated NOAA fixture under both settings. File: read.sheet.R, write.sheet.R, man/read.sheet.Rd, man/write.sheet.Rd, tests/testthat/test-read.sheet.R, tests/testthat/test-write.sheet.R ----------------------------------------------------------------- - read.sheet() and write.sheet() gain long = TRUE, for "long" or "tidy" files of series, year, value triples. Both sides gained it together rather than only the reader. Long is the layout that arrives FROM somewhere else -- a database export, a collaborator's script -- so reading it is the half that earns its keep, and pivoting one into a valid rwl is the step people get wrong. But with a reader and no writer the long path could only have been tested against hand-built fixtures, which tests the fixture as much as the code. With both, ca533, co021 and nm046 each survive write.sheet(long = TRUE) followed by read.sheet(long = TRUE) identically, the same way they do wide. Long columns are taken by position, which is what write.sheet() emits. If the header names all three of series, year and value, in any case and any order, the names win instead: a file from elsewhere need not be in our order. Anything in between is position. On read, the year span runs from the earliest to the latest year in the file, so a year no series measured comes back as a row of NA and is reported -- years with no data at all break several dplR functions, and in the wide layout that same file would have carried an explicit blank row. One series may hold only one value per year; in the wide layout that is impossible by construction, here it is checked. - A note on what long format can and cannot represent, because the obvious statement of it is wrong. A year in which every series is NA has no rows in a long file, but only a LEADING or TRAILING such year is actually lost: an interior one survives, because the reader rebuilds the span from the earliest and latest year present and the year reappears as a row of NA exactly where it was. Only an edge year has nothing outside it to pin the span. write.sheet(long = TRUE) therefore warns for edge all-NA years and stays quiet for interior ones, rather than warning about every case and crying wolf on the ones that round trip perfectly well. File: write.sheet.R, write.rwl.R, man/write.sheet.Rd, man/write.rwl.Rd, tests/testthat/test-write.sheet.R ----------------------------------------------------------------- - There is a new writer, write.sheet(), the other half of read.sheet(). write.rwl() gains format = "csv", which dispatches to it and, like every other branch of that switch, returns fname. A csv has no format specification to conform to, so every default is a decision rather than a reading of somebody's spec. The one that earns its keep is prec. A ring width read at 0.001 mm is a double, and writing it with R's default formatting puts 0.5670000000000001 in the file -- ugly, and a lie about how precisely the ring was measured. prec = NULL reads the precision back off the provenance record that read.sheet() and read.tucson() attach, falling back to inferring it from the data. This is the first thing to make practical use of that record. A derived precision is checked against the data before it is used and a given one is not, which is the right way round and easy to get backwards. rwl.granularity() rounds to three decimals before taking its gcd, so on an object holding something finer it answers 0.001 and rounding there would discard real digits; provenance can be stale for the same reason, since it describes the file the object was read from and the object may have been through arithmetic since. Neither is an instruction from the caller, so if rounding would lose data the file is written at full precision with a warning. A prec passed explicitly IS an instruction and is obeyed: prec = 0.1 rounds. Unlike write.tucson(), prec is not restricted to 0.01 and 0.001 -- that restriction belongs to the decadal format's six-column field and two-valued flag, and a csv has neither. A ring width of zero is written as a zero and a missing value as an empty cell. Those are different statements about a tree in a year and are never written the same way. Trailing zeros are kept, so at prec = 0.001 a width of 0.51 is written 0.510. Series ids are written verbatim, quoted with the ordinary csv escape if one contains the separator or a quote. The wide round trip is exact: ca533, co021 and nm046 each survive write.sheet() followed by read.sheet() identically. Provenance is dropped from both sides before comparing, not just from the object that came back -- the stored datasets carry a record of their own, and in any case provenance describes the file an object was read from, so a fresh read of a freshly written file legitimately has a different path and a different reader. The data is what must survive. This release writes the wide, comma separated layout. Long format and separators other than the comma refuse, matching read.sheet(): a tab file that this package cannot read back is worse to ship than a clear refusal. File: read.sheet.R, csv2rwl.R, read.rwl.R, man/read.sheet.Rd, man/read.rwl.Rd, tests/testthat/test-read.sheet.R ----------------------------------------------------------------- - There is a new reader for spreadsheet-shaped ring width files, read.sheet(). Years down the rows, series across the columns, years in the first column -- the layout people actually have in Excel. It replaces csv2rwl(), which is deprecated and now forwards to it. The problem with csv2rwl() was not that it was small; it was that it was unverified. It set class "rwl" directly, which routed around as.rwl() and its requirement that the row names be consecutive integers, so it accepted files and returned objects that were wrong in ways nothing downstream detects: a year column running 1901, 1902, 1905 was taken whole and time() then reported a discontinuous span for the life of the object; a stray text column was carried into the rwl and sat there until some later function died on it, far from the cause; and series ids went through read.table(check.names = TRUE), so "1A" came back as "X1A" and "LF-2B" as "LF.2B", silently. Series ids are data, and a reader that alters them without saying so is losing data. read.sheet() therefore checks more than read.tucson() does, not less: a Tucson file at least has a format to violate, while a sheet has no discipline beyond what the person who saved it happened to do. Years must be whole, ascending, unique and consecutive; every column after the first must be numeric, and offending cells are named rather than the whole column being called text; ids are read verbatim, with a stripped byte order mark or a resolved duplicate reported and recorded rather than done quietly. Duplicated ids are renamed and recorded, as read.tucson() does, instead of stopping. It returns the same nine-field provenance record as read.tucson(), in the same shape and with the same event ids where a condition means the same thing, so nothing downstream has to ask which reader made an object. Precision is inferred from the data rather than read off a flag, since a sheet carries none. A NOAA/NCEI template file is refused rather than half-read. Those are a commented metadata block above a tab separated data table, and that table is exactly the shape this reader is built for -- so a reader that skipped comments in the ordinary way would parse one and silently discard the coordinates, species, investigators and DOI. The refusal is in place now, before tab support arrives, because it has to be there before the parsing can reach those files rather than after. This release reads the wide, comma separated layout. Long format and separators other than the comma refuse with a message naming what they are waiting on, rather than doing something adjacent. - csv2rwl() is deprecated as of 1.8.0. It warns on every call rather than once per session, which is not lifecycle's default: this deprecation changes what the function accepts, not just its name, so someone reading a directory of sheets in a loop would otherwise get one warning and then silence while individual files were refused. A file that csv2rwl() used to read may now stop with an error. That is the deprecation working -- the object it returned for those files was invalid. There is deliberately no csv2rwl.legacy(): read.tucson.legacy() exists because the old reader's behaviour is different and occasionally wanted, which is not the case here. - read.rwl() now calls read.sheet() for format = "csv" and for csv files found by format = "auto", so a user who never typed csv2rwl() is not told it is deprecated. File: Extract.rwl.R, man/Extract.rwl.Rd, tests/testthat/test-Extract.rwl.R ----------------------------------------------------------------- - There is now a `[` method for rwl objects. Subsetting used to fall through to `[.data.frame`, which kept the class and then did two things wrong with it, and left a third undone. It dropped the provenance record. read.tucson() attaches what it saw to the object, and any attribute `[.data.frame` does not know about goes on a column subset, so cutting a collection down to a few series threw away the header, the precisions, the gaps and, worst of all, the renames -- which exist nowhere else, so a subset holding a renamed series carried an id that appears in no file and nothing left to say so. The record now travels with the data and is cut down to fit: series-level rows are kept for the series that remain, gaps outside the years that remain are dropped, mixed.precision is recomputed, and a subset element records the series and the first and last year in hand, with all.series saying whether the file's full set of series is still there. Anything that cannot be matched by name is dropped rather than carried, so a record never describes series that are not in the object. And it did not check the years. An rwl object promises one row per year in order, and every function that reads a date off a row name -- time(), plot(), detrend(), chron(), rwl.stats(), the crossdating functions -- assumes the next row is the next year. ca533[c(1, 5, 9), ] returned an object of class rwl whose years were 626, 630 and 634, and all of those then read it as three consecutive years beginning in 626, silently. When row subsetting leaves years that are not consecutive and increasing -- rows taken from the middle, repeated, or reversed -- the result is now returned as a plain data.frame with a warning saying what it would have broken. No values change; what is withdrawn is the claim that the result is a set of dated series. A run of years, which is what head(), tail(), subset() and common.interval() produce, is unaffected and stays an rwl object. - Selecting series now trims the years that the remaining series do not cover. Collections are ragged, so a subset of the series nearly always leaves empty years at one or both ends: ca533[, 1:3] used to span 626-1983, because one of the 31 series it dropped reached back to 626, and every summary of the result then described a span that nothing in it was measured over. Only the leading and trailing empty years go; an empty year inside the span stays, since taking it out would leave the years no longer consecutive. A subset in which nothing at all is measured is left whole rather than reduced to nothing, and rwl.check() reports it. The trimming happens only where a series actually went away and the call does not index rows. Name years -- x[i, ], x[i, j], head(), common.interval() -- and exactly those rows come back, empty or not: a year window is how an rwl object gets lined up against something else, and quietly returning fewer rows than were asked for would put that alignment out by however many years happened to be empty at the edge. A call that keeps every series, such as the reordering x[, order(...)] that xdate.floater() does before it draws its segment plot, has dropped nothing to trim for and leaves the years alone. Drop series and you get the years the rest of them cover. A single series taken with drop = TRUE is a numeric vector, which carries no years, so it is returned at full length and still lines up with the object it came from; drop = FALSE gives a trimmed one-series rwl object. common.interval() is unaffected and is still the way to cut a collection down to the years its series share, which is a different question from the years they cover between them. - There is a subset() method for rwl objects. subset.data.frame() worked already -- it ends in x[r, vars, drop = drop], which reaches the method above -- but it always passes a row index, so subset(x, select = 1:3) would have kept the empty years that x[, 1:3] trims. The method passes a row index only when the call made one, so the two agree. File: data/ca533.rda, data/co021.rda, data/nm046.rda, data/wa082.rda, man/ca533.Rd, man/co021.Rd, man/nm046.Rd, man/wa082.Rd ----------------------------------------------------------------- - The four data sets that come from the ITRDB now carry the provenance record described below, so they name the archived file they were read from, the precision they were measured at, and, for wa082, the one interior gap the old reader filled with a zero. The measurements are unchanged: each was checked against its archived file first, and the record was attached to the shipped values rather than replacing them. Each help file gained an example showing how to read it. anos1, gp.rwl and zof.rwl are not from the ITRDB and are unchanged. Three of the four ITRDB files carry no header at all -- ca533, co021 and nm046 begin at the first data line -- so their record has an empty header, which is what the files say. File: read.tucson.R, man/read.tucson.Rd ----------------------------------------------------------------- - read.tucson() now records what it saw and attaches it to what it returns, as attr(x, "dplR.provenance"). It holds the header lines, the precision each series was measured at and whether the file mixes them, any series the reader renamed as old and new, the interior gaps and what the file held at each, and one row per recoverable problem found while parsing, each under a stable event id. All of this was computed and thrown away. Everything the reader noticed became a message and then nothing, so rwl.check() had to reopen the file to recover a fraction of it, and the parts that are not in the file in any readable form could not be recovered at all. The clearest of those is renaming: where a file gives two cores the same id, the reader renames one, and after that the object carries a series name that appears nowhere in the file, with nothing to say so. It is an attribute rather than a changed return value, so read.rwl() and existing callers are untouched. R drops it on column subsetting and in functions such as detrend(), which is the safe direction: provenance describing 43 series would be wrong on a 3-series subset, so it goes absent rather than stale. - The interior-gap runs are now worked out whether or not verbose is set. They were built inside the verbose branch, so read.tucson(verbose = FALSE) would have returned a provenance record claiming the file had no gaps. A record must not depend on a display setting. - No timestamp is recorded. Stamping the read time was tried and removed: it made two reads of one file unequal, so all.equal() on the objects stopped working and any comparison of a fresh read against a stored one always showed a difference. File: caps.R, man/caps.Rd ----------------------------------------------------------------- - caps() now stops when given a series containing NA. It used to carry the NA through the spline and return a full-length vector of NA, which looks like an answer and is not: a caller dividing by it gets a series of NA and no indication why. A smoothing spline through a series with missing values is not defined, and the caller is the one who has to decide what to do about the gap. This was documented behaviour, so the help file and its example have been updated. - The caps() example still described the default as a 2/3 spline and computed 2/3 of the series length for its legend, though the default became 32 in version 1.7.8. It now says 32. File: rwl.check.R, rwl.report.R, NAMESPACE, man/rwl.check.Rd, tests/testthat/test-rwl.check.R ----------------------------------------------------------------- - New function rwl.check(), with a new class of the same name, plus rwl.check.control(), rwl.check.catalogue(), and print, summary and as.data.frame methods. Where rwl.report() describes a collection for a reader, rwl.check() looks for defects and returns them as rows, so that examining one file and sweeping an archive are the same operation. as.data.frame() gives one row per finding; summary() gives one row per file with a count column per check, and those rows rbind across files into a triage table. Every finding carries a stable check ID and a severity of error, warning or note. rwl.check.catalogue() lists them. The IDs are meant as a contract: they are what a sweep filters on and what two runs are compared by. No check can stop the report. rwl.report() dies outright on a single-series collection, on a zero-variance series, and on series that do not overlap, because interseries.cor() throws; those are exactly the files worth looking at. Every check in rwl.check() runs inside tryCatch() and a check that fails becomes a finding. A "provenance" group reads read.tucson()'s record and reports what no amount of looking at the file or the data could show, including RWL_ID_RENAMED, RWL_MIXED_PRECISION, RWL_YEAR_CLASH and RWL_COLUMN_LAYOUT. The group is silent for an object that carries no such record, including one read by read.tucson.legacy(). Checks that nothing in dplR previously made: series misdated by a whole number of years (found by cross-correlating each series against a master over a range of lags); a series archived twice under two IDs; constant stretches standing in for measurements; series ids that break the file's dominant pattern or carry a different site code; and, when given a path rather than an object, line endings, stray tabs, non-ASCII bytes, and the span declared in the ITRDB header against the span the measurements actually cover. The crossdating checks run on data high-pass filtered with caps() at the COFECHA stiffness of 32 years, so they agree with corr.rwl.seg() and detrend(method = "Spline") rather than inventing their own idea of a common signal. Correlating raw ring widths measures the agreement of age trends instead: on wa082 the raw and filtered measures agree at only r = 0.15, and filtered, the correlation tracks interseries.cor() at r = 0.90. A series with a gap inside its measured span cannot be detrended and is reported as RWL_UNCHECKABLE rather than being filtered across the gap, which would fit the curve to a series that was never measured. They also ask whether a series fits the collection it is in rather than whether it clears a fixed correlation, because what counts as a good correlation is a property of the site. Over a 1,000-file ITRDB sample a fixed cut at 0.2 spent 44% of its flags on series that were merely in low-correlation collections, and said nothing about 556 obvious outliers in 301 files. Judged against its own collection the check flags a comparable number of series but no longer depends on series length, and fires more often in well-dated collections than in loose ones, which is the right way round. A collection with little common signal is reported once, as RWL_WEAK_COLLECTION and as a note. Some collections legitimately have little of it -- ecological samples taken for growth rather than for a climate signal -- so it is not a fault, but it decides whether the series can be crossdated or carry a chronology. Thresholds were calibrated on a 1,000-file sample of the ITRDB. At the defaults 3.8% of files raise an error, 36% raise a warning and 62% are clean. File: man/wa082.Rd ----------------------------------------------------------------- - Documented that wa082 holds a zero for series 712011 in 1900, where the source file records -999, i.e. no measurement. The data set was built with the old reader, which filled interior gaps with zero. The values are unchanged, for continuity; the help file now says so and notes that read.tucson(fill.internal.NA = 0) reproduces them. The other data sets were checked against their sources at the same time. ca533, co021 and nm046 contain no interior gaps at all, so the reader change does not affect them, and all three still match their archived files exactly. anos1 and zof.rwl contain no zeros and so cannot be carrying a filled gap. gp.rwl is not from the ITRDB and has no file to check against. File: rwl.report.R, man/rwl.report.Rd ----------------------------------------------------------------- - rwl.report() says when a file carried no header rather than omitting the site line. Three of the four ITRDB collections shipped with dplR begin at their first data line, and silently dropping the line read as though the site were unknown or as though something had failed. - rwl.report() no longer says interior gaps were filled when the file has no interior gaps. The fill setting is recorded whether or not it did anything, and reporting it unconditionally stated something untrue about the data. - rwl.report() opens with a header naming the file, the site and species from the ITRDB header, the declared precision, and the span the header claims -- worth seeing beside the span the data covers, since cana209 declares 1459-1960 and measures 1713-2001. It also notes when the reader renamed a series, so the ids in the report are not the file's, and when interior gaps were filled. It is taken from the provenance record, so an object without one is reported exactly as before. File: read.tucson.R ----------------------------------------------------------------- - The header kept in the provenance record is taken from the lines as read, not after truncation at column 72. The ITRDB header carries its declared span at about columns 68 to 77, so truncating lost the end year: cana209's header came back reading 1459 with no second year, which was the one thing it was being kept for. File: rwl.report.R ----------------------------------------------------------------- - RWL_INTERNAL_NA carries the consequence, not just the fact. An interior gap is not a fault -- the file records no measurement for those years, which is worth keeping -- but it decides what can be done next, so the finding names the functions that will fail on it and points at fill.internal.NA(). It is reported once per series rather than once per gap. - rwl.report() failed to report internal NA values, missing rings, consecutive zeros, and small and large rings whenever every series in the object produced the same number of hits. The per-series lists were built with apply(), which simplifies its result to a vector or matrix in that case, and the years were then recovered from names that simplification had discarded. The commonest instance was the plainest one: a single series with a single internal NA, where every series returns one value, which reported "None". Rebuilt with lapply() and an explicit year lookup. - rwl.report() reported allZeroYears as row indices rather than years. On co021 it gave 320, 331, 409 and 410 for the years 1495, 1506, 1584 and 1585. - The count of missing rings was NA, printing "Number of missing (0) rings: NA (NA%)", for any collection with no zeros at all. It was read out of a table() by name, and the name is absent when the count is zero. cana209 is such a file. - Fixed the run-start index in the consecutive-zeros helper, which was wrong for a run beginning at the first ring of a series. File: read.tucson.R, read.tucson.legacy.R, read.rwl.R, NAMESPACE, DESCRIPTION ----------------------------------------------------------------- - Replaced read.tucson() with a new implementation, originally written by Hung Nguyen and reworked by Andy Bunn. The previous reader is retained unchanged as read.tucson.legacy(), following the naming of glk.legacy() and rwi.stats.legacy(). read.rwl() routes to the new reader for format = "tucson" and for auto-detection. - IMPORTANT: interior gaps are no longer filled with zero. Where a file records no measurement for a stretch of years inside a series, the old reader silently wrote zeros in the C readloop. A ring width of zero means a locally absent ring, which is a real observation about a tree in a year; a gap means the ring was not measured, which is a statement about the data. The reader no longer substitutes one for the other. The new fill.internal.NA argument controls this, and fill.internal.NA = 0 reproduces the old values exactly. Note that detrend() and other functions that reject NA inside a series will fail on data with interior gaps. Such gaps must now be filled deliberately rather than by the reader's default. - The new reader also: reads files the old one refuses; reports malformed lines rather than guessing at them, returning a line as NA when its fixed-width and whitespace readings disagree; reports repeated series IDs and renames them when their records overlap, instead of silently welding them into one series; expands tab characters to 8-column stops before measuring any column position; and determines the header and the column layout per line rather than per file. - Determining the column layout per line means years before -999 and 8-character series IDs can occur in the same file, which the old reader could not represent: it used the per-file 'long' switch, and a mixed file was read wrongly whichever way that switch was set. - New arguments: comment.char, fix.dup.char, fill.internal.NA, strict. strict = TRUE turns every recoverable problem into an error so that a pipeline can refuse a file rather than carry a warned guess forward. - The header, long and encoding arguments are accepted and ignored, and warn when supplied, so that existing code and read.rwl()'s pass-through of ... do not fail with an "unused argument" error. The first six arguments keep their old positions, so positional calls still bind correctly. encoding is a genuine gap rather than merely an unused argument: the new reader has no encoding handling and reads the file as it stands. read.tucson.legacy() still honours it. - Columns are returned in the order the series appear in the file. - Added data.table to Imports. The reader is built on it, and the importFrom(data.table, ...) block in NAMESPACE is load-bearing: data.table's cedta() check decides whether `[.data.table` dispatches correctly, and it answers on the basis of a package's imports. Remove that block and every := in read.tucson() breaks silently. File: write.tucson.R -------------------- - write.tucson() now writes interior NA faithfully, so that a read.tucson() -> write.tucson() -> read.tucson() cycle preserves the gaps instead of re-introducing exactly the zeros the new reader's default removes. Years between a series' first and last measurement where the data hold no value are written as -999, the negative value the Tucson format uses to mark missing data, at both precisions. This is what prec = 0.01 already did; prec = 0.001 wrote 0 there, which claims a locally absent ring where the data said the ring was not measured. Files with no interior NA are written byte-for-byte as before, at both precisions. Note that read.tucson.legacy(), and other dendrochronology programs, read the sentinel back as a ring width of zero. A message is given whenever gaps are written. - New argument fill.internal.NA, the same argument with the same meaning and the same NULL default as in read.tucson(). NULL invents nothing; anything else is passed to fill.internal.NA() before the file is written, so fill.internal.NA = 0 gives the old prec = 0.001 output and "Mean", "Spline" and "Linear" interpolate. - A series that is entirely NA is now skipped with a warning naming it. Previously it produced "'from' must be a finite number" from inside seq(), after the preceding series had already been written to the file. - Series IDs keep their hyphens and underscores. write.tucson() removed every character outside a-z, A-Z and 0-9, which is dplR's rule and not the format's: the ITRDB description names only the columns a series ID occupies and says nothing about its character set. Hyphens and underscores are common in real IDs, do not disturb the fixed-width layout, and are read back unchanged by read.tucson(), which read CC11-3 and wrote CC113. The removal was not merely cosmetic. Shortening a name can collide with a name that needed no change, and the duplicate pass then renames both: a file holding CC1-1 and CC11 was written out as CC110 and CC111, neither of which is an ID in the input. New argument extra.chars names the characters allowed beyond alphanumerics, defaulting to "-", "_" and ".". extra.chars = character(0) restores the old rule. Whitespace and control characters are still removed, and "#" is refused outright because it is read.tucson()'s default comment.char and would make the whole line disappear. - One position is exempt: a hyphen may not be the 8th character of a series ID, because a minus sign in column 8 is what tells a 5-character year before -999 from an 8-character ID. Such an ID is shortened by that hyphen, with a warning. Only reachable with long.names = TRUE. File: helpers.R --------------- - fix.names() gained an extra.chars argument, empty by default, naming characters to allow beyond a-z, A-Z and 0-9. write.tucson() passes "-", "_" and "."; write.compact() keeps the default. Characters that cannot be placed safely in a bracket expression for both regex engines the function uses, along with whitespace and control characters, are refused. - fix.names() now reports a renaming once, not three or four times. The character-removal, over-length and duplicate conditions each used to raise their own warning as they were found, so a single renamed series could produce three warnings that between them never said which series was renamed or what it became. They are now collected and reported in one warning that gives both the reasons and the first few renamings as old -> new, and points at mapping.fname for the full list. That was least helpful in the case that matters most: a rename that collides drags in a series the caller never touched, and that series then leaves under a name appearing nowhere in the input. The three old message strings are gone, so their Finnish translations need regenerating. File: write.tucson.R -------------------- - An all-NA series is reported once for the whole file rather than once per series. A data frame left empty by subsetting can have dozens of such columns. File: read.tucson.R ------------------- - A stop marker now ends a record. Previously read.tucson() inferred that one series ID held more than one record from file order alone: a new block began only where the ID changed or the decade label failed to advance. That caught a second record written out of order and missed an identical one written in order, so the same structure was reported in some files and passed over in silence in others. ut542 shows both shapes: of eight series holding a mid-record -9999 followed by more data under the same ID, only six were reported. The stop marker is the format's own statement that a record has ended, so it is now what cuts the block. This changes no values and merges nothing differently -- ut542 returns an identical object -- but the "entered as N separately terminated records" report now fires consistently. - Reworded that report. Entering one core as several terminated records with gaps between them is allowable and usually deliberate; the old wording said the reading was wrong, which overstated it. - The verbose interior-gap list now prints one line per series rather than one per gap, so its length matches the series count in the header above it; the individual gaps are still given, on the series' own line. It is also ordered to match the columns of the returned object rather than alphabetically, so the list reads in the same order as the data. File: helpers.R --------------- - Softened the internal-NA warning in check.rwl(). It described internal NAs as "not standard practice", which the new read.tucson() default now contradicts: a gap where a file records no measurement is the correct result, not a deviation. File: test-read.tucson.R, test-io.R ----------------------------------- - Added a test suite for the new reader. The existing read.tucson tests in test-io.R were written against the old reader's behaviour and now target read.tucson.legacy(). File: helpers.R, NAMESPACE -------------------------- - Added check.rwl(), a new exported function that validates an object as a proper rwl. If the input is not already class "rwl", coercion is attempted via as.rwl(): on success a warning is issued; on failure a single informative error is raised that describes both the class problem and the structural reason coercion failed. After class validation, check.rwl() warns about any internal NA values (NAs sandwiched between real values within a series) and names the affected series, directing the user to fill.internal.NA(). File: as.rwl.R -------------- - Added a check that all columns are numeric. Previously as.rwl() would accept a data.frame with character columns and silently assign the rwl class to an invalid object. File: bai.in.R, bai.out.R, ccf.series.rwl.R, cms.R, common.interval.R, corr.rwl.seg.R, corr.series.seg.R, detrend.R, i.detrend.R, interseries.cor.R, pointer.R, rcs.R, rwl.stats.R, seg.plot.R, series.rwl.plot.R, spag.plot.R, ssf.R, strip.rwl.R, xdate.floater.R ----------------------------------------------------------------------- - Replaced ad-hoc rwl validation (bare is.data.frame() checks or warning-only class checks) with check.rwl() at the top of each function, providing uniform, informative validation across all public functions that accept an rwl object. File: sgc.R, sgc.Rd -------------------- - Renamed the first argument from x to rwl for consistency with the rest of the package. Updated documentation accordingly. File: chron.R, chron.Rd ------------------------ - Renamed the first argument from x to rwi to correctly reflect that chron() expects detrended ring-width indices, not raw ring widths. Updated documentation accordingly. File: chron.ars.R, chron.ars.Rd --------------------------------- - Renamed the first argument from x to rwi for the same reason as chron(). Inner helper functions (pooledAR, postAR, prewhitenAR, prewhitenARIMA) retain their own local x arguments, which are unchanged. Updated documentation accordingly. File: tests/testthat/test-check.rwl.R -------------------------------------- - Added 17 new tests covering check.rwl() and as.rwl(): happy path, coercion with warning, structural failure messages, internal NA detection, and all as.rwl() error conditions. * CHANGES IN dplR VERSION 1.7.9 File: chron.ars.R ---------------- - Fixed a bug in the pooledAR helper where three guard conditions used break instead of next, silently discarding series pairs with no temporal overlap and producing incorrect ACF estimates, wrong AR order selection, and biased chronologies. Also fixed three related edge cases: prewhitenAR and prewhitenARIMA now return the series unchanged when AR order is zero; postAR now returns the series unchanged when passed an empty coefficient vector; chron.ars now stops with an informative message when firstAICmin=TRUE and pooled AIC decreases monotonically without reaching a local minimum. File: rcs.R, rcs.Rd ---------------- - Fixed a bug where the rwca matrix was allocated using the full row count of rwl rather than the length of the longest series, causing indexing errors with floating series. - Added method argument accepting "caps" (default) or "ads" to fit the regional curve using an age-dependent spline via ads. Added pos.slope argument passed through to ads. Default for nyrs now differs by method: 10% of RC length for "caps", 50 for "ads". - Added min.n argument to truncate the regional curve at the last cambial age where sample depth meets or exceeds the threshold. - Removed rc.in and check arguments. Note: breaking change for code that used either argument. - po now defaults to NULL. When NULL, a pith offset of 1 is assumed for all series. - Updated documentation throughout. File: xdate.floater.R, xdate.floater.Rd ---------------- - Major refactor. Function now returns a floater S3 class object. - Added print.floater() and plot.floater() S3 methods. - plot.floater() now includes an interseries correlation quantile envelope (5th-95th percentile of master) and median interseries correlation line. - Changed default for return.rwl from FALSE to TRUE. - Results are now filtered to exclude future dates. File: universalPOWT.R ---------------- - Fixed a bug where a non-positive power estimate would cause X^p to return nonsensical results. Function now falls back to log transformation when p <= 0, consistent with the Box-Cox limiting case. A warning is issued when the fallback is triggered. Hat tip to Stefan Klesse. See https://github.com/OpenDendro/dplR/pull/32 File: bakker.R ---------------- - Fixed a bug where BAI was miscalculated in the first row because diff() returns n-1 values, leaving the first year unassigned. Fix prepends pi*r0[1]^2 as the first year's BAI, consistent with assuming pith radius of zero. Hat tip to Stefan Klesse. See https://github.com/OpenDendro/dplR/pull/33 File: rwi.stats.running.R ---------------- - Added a stop() when rwi has a non-positive grand mean, which causes normalize1() to return incorrect rbar and eps values. This can occur when rwi has been detrended using difference=TRUE. Hat tip to Stefan Klesse for identifying the bug. See https://github.com/OpenDendro/dplR/pull/34 File: ads.Rd ---------------- - Fixed description of pos.slope argument: behavior is triggered when pos.slope=FALSE, not TRUE. Fixed several typos. See https://github.com/OpenDendro/dplR/pull/35 File: ads.Rd ---------------- - Fixed description of pos.slope argument: behavior is triggered when pos.slope=FALSE, not TRUE. Fixed several typos. Hat tip to SK for the catch. See https://github.com/OpenDendro/dplR/pull/35 File: sss.Rd, rwi.stats.running.Rd ---------------- - Updated sss help to clarify the frequently mis-cited EPS=0.85 threshold from Wigley et al. (1984) following discussion with Stefan Klesse and Allan Buras. - Added Wigley, Briffa and Jones (1984) to references. - Reformatted details section for readability. File: Various man files ---------------- - Minor documentation updates across multiple help files (ads, caps, ccf.series.rwl, chron.stabilized, csv2rwl, detrend.series, read.crn, read.tucson, rwl.report, ssf, treeMean, write.crn, write.tucson). * CHANGES IN dplR VERSION 1.7.8 File: powt ---------------- - Added SK's universalPOWT function and condensed the existing cook powt function into a single function with a method call. File: rwl.stats ---------------- - Added kurtosis to summary stats File: glk ---------------- - Man file updated to improve example. See https://github.com/OpenDendro/dplR/issues/26 File: detrend.series ---------------- - Small change to Ar method to get mean right. See https://github.com/OpenDendro/dplR/issues/22 File: read.fh ---------------- - Added an extra check for the end and start date, in case there is an error in the FH-file, with the end date before the start date. File: ffcsaps ---------------- - ffcsaps is deprecated as it has been replaced with caps. Will be defunct after 1.7.8 At the moment ffcsaps calls caps using the lifecycle::deprecate_warn() File: dplR.c .h, redfit.c ---------------- - Resolved R CMD check NOTE by using a well documented, conditionally backported function instead of a non-API entrypoint * CHANGES IN dplR VERSION 1.7.7 File: DESCRIPTION ---------------- - Changing the home of the repo to opendendro's GH File: bakker.R .Rd, zo.rwl, zo.anc ---------------- - Added SK's bakker func File: read.tucson.R ---------------- - Added a verbose flag File: rwl.report.R ---------------- - Added in years with all zeros in the report. And a warning that it can break dplR. - Removing plyr as an import -- very simple fix to get rid of alply - Setting a new ouptput for consecutive zeros. Those make me nervous. - Removing invisible so the rwl report can be useful as an object. - Adding a new check for unconnected flaoters - Changing arg names to better match dplR conventions File: ssf.R ---------------- - Found a bug where input data with rows of all zero caused div0 problems. Added a check to stop program if input has all zero rows. - Removing pos.slope as an argument and hardcoding to TRUE. - Changed default to caps rather than ads. - Added a difference option for making rwi. - Changed output so that iter0 is incuded in the output arrays when return.info is true File: caps.R ---------------- - Added arg so that nyrs can be between 0 and 1 and then treated as a %. Changed default to be 2/3 rather than 1/2 of n * CHANGES IN dplR VERSION 1.7.6 File: caps.R ---------------- - Added a % option to the spline so that values of nyrs between 0 and 1 are a percentage of the length of y. - Changed the default otion for nyrs to a 2/3 spline * CHANGES IN dplR VERSION 1.7.5 File: read.compact.Rd ---------------- - Added an example. File: read.compact.c ---------------- - warning on # of args in two spots. Looked at it after email from Kurt H File: detrend.series.Rd ---------------- - missing \ in man page File: adsf.f95 and capsf.f95 ---------------- - Replaced DFLOAT with DBLE as per BDR's reasonable demand. File: ssf ---------------- - Changed output to be class(crn) for list with long output File: chron.ci ---------------- - Changed output to be class(crn) File: crn.plot and plot.crn ---------------- - Finally reconciled the misnaming of crn.plot vs plot.crn etc. The new default plot method is simpler and cleaner. Help pages updated and a warning crn.plot is called directly. File: chron.stabilized ---------------- - Added class crn to returned value as per github issue #18 File: read.fh ---------------- - Ron Visser added a BC correction to the function. File: read.rwl ---------------- - Edited function to look for commas (",") in lower blocks of data as opposed to the header File: csv2rwl ---------------- - Added a stop error and message when there are duplicate series IDs File: read.tucson ---------------- - Changed error message when there are duplicate series IDs to be more informative File: normalize1 ---------------- - Found a bug so that when hanning was used (in say corr.rel.seg) a div0 error could occur resulting in a Nan that got propogated forward. Dealt with in the same way as detrend.series by setting 0 to 0.001. File: powt.series ---------------- - Adding function so that a power transform can be done on a single series alone * CHANGES IN dplR VERSION 1.7.4 File: chron.ci ---------------- - New function that calculates conf intervals for a chron via boot File: detrend.series ---------------- - Fixed bug with method Ar using difference. - Added in a flag for dirtyDogs to return.info * CHANGES IN dplR VERSION 1.7.3 File: Various ---------------- - Replaced ffcsaps with caps in most instances across the package. - Removing prefix argument from chron. - Removing all vingettes and pointing to Learning to Love dplR on startup - Typos and code cleanup in several files File: zzz ---------------- - Adding startup message when dplR is loaded File: crn.plot ---------------- - Added crn names to plot. helpful for ars crn and var stab crn plotting. File: ssf ---------------- - A simple signal free implementation. File: chron.ars ---------------- - The ARSTAN chronology at long last. File: detrend ---------------- - When dopar is used for the looping, the series name wasn't being passed in. So modified code to take the series name. I think I did it marginally right. But dopar, foreach, and do.call are all prety opaque to me. File: detrend.series ---------------- - Went to heroic efforts to improve the warnings that the user sees when there are negative fits from detrending. It was a huge time sink and hassle. File: nm046 ---------------- - Added new data file, nm046 is a rwl file that has a dirty dog in it (one where spline fits are negative in detrending). File: ffcsaps ---------------- - Added `ties = "ordered"` to approx() to avoid warning thrown by regularize.values File: caps ---------------- - Added new function caps() for Cook and Peters Spline. This is a function to calculate the classic ARSTAN spline. The work is done in Fortran and involved a C wrapper for the Fortran code. As a result the new files are caps.R, caps.Rd for the main files and the help function. But there are new Fortran and C files (caps.f95, caps.c) and changes to init.c and registered.h. Oh and NAMESPACE too. Note that this function is 1000s of times faster than ffcsaps. File: ads ---------------- - Added new function ads(). This is a function to calculate age-dependent splines that came via Ed Cook. The work is done in Fortran and involved a C wrapper for the Fortran code. As a result the new files are ads.R, ads.Rd for the main files and the help function. But there are new Fortran and C files (adsf.f95, adsc.c) and changes to init.c and registered.h. Oh and NAMESPACE too. File: bai.out, bai.in ---------------- - Added check to make sure `d2pith` and `diam` are data.frames and not, say, tibbles File: ca533.Rd, cana157.Rd, co021.Rd, wa082.Rd ---------------- - Updated access dates and changed ftp to https per email from BDR on April 20 2021 * CHANGES IN dplR VERSION 1.7.2 File: glk.R sgc.R ---------------- - Revising glk for speed and adding sgc as a new function from Ronald Visser File: chron.stabilized.R ---------------- - Adding variance stabilization from Stefan Kleese see https://github.com/AndyBunn/dplR/issues/3 File: time.rwl.R ---------------- - Adding `time<-` as a method for `time.rwl<-` and `time.crn<-` so that times can be set as well as retrieved from rwl and crn. Getting the NAMESPACE, and .Rd files right was unpleasant. File: powt.R ---------------- - C Zang added rescale arguemnt to powt to have results align better with ARSTAN output. * CHANGES IN dplR VERSION 1.7.1 File: ccf.series.rwl.R ---------------- - Fixed stringsAsFactors issue pointed out by Kurt H File: xdate.floater.R and .Rd ---------------- - New function to cross date a floating series (i.e., one with no dates). * CHANGES IN dplR VERSION 1.7.0 File: insert.ring.R and delete.ring.R ---------------- - Adding an argument fix.length to maintain the length of the input and output. This makes it much easier to edit series and insert them back into a rwl object as needed. File: plot.crs.R ---------------- - This is a new S3 method for plotting the ouptut from corr.rwl.seg. The rationale for this is to reduce computational overhard in the shiny app I'm developing for xdating. File: corr.rwl.seg.R ---------------- - Added class crs to the output object as well as added some new items to the list that are returned (e.g., the rwi, seg legnths and so on). This is done so that I could make a new S3 method for plotting objects `plot.crs()`. New output ites are explained in man file. The plotting code if make.plot=TRUE is gone and replaced with a call to plot.crs. File: redfit.R ---------------- - Fixed bug where the handling of series with duplicate times (ages) would produce an error. Thanks to Mark Hall for reporting. File: plotRings.R and .Rd ---------------- - Making plotRings defunct as there were too many issues with the animation library and especially ImageMagick integrating smoothly for some users. Suggested to Darwin that he make plotRings a stand alone library. File: xdate-dplR.Rnw ---------------- - Rewritten to explain new args in ccf. File: xskel.ccf.plot.R and .Rd ---------------- - Added new argument to allow user to specify whether a series or master gets passed to ccf as x. The convention was screwy. File: ccf.series.rwl.R and .Rd ---------------- - Added new argument to allow user to specify whether a series or master gets passed to ccf as x. The convention was screwy. File: series.rwl.plot ---------------- - Cosmetic changes to plot File: intro-dplR ---------------- - typos * CHANGES IN dplR VERSION 1.6.9 File: sss.R and .Rd ---------------- - Adding subsample signal strength as a stand alone function File: spag.plot.R and .Rd ---------------- - Small bug fix File: crn.plot.R and .Rd ---------------- - Small bug fix File: as.rwl.R and .Rd ---------------- - Adding a convenience function to transform data.frame or matrix to class rwl File: powt.R and .Rd ---------------- - Small change so that powt returns class rwl as well as data.frame File: pass.filt.R and .Rd ---------------- - Adding a wrapper function for signal::butter and signal::filtfilt to get low-pass, high-pass, band-pass filtering implemented as per a user request. File: rwl.stats.R and .Rd ---------------- - Removing sens1 and sens2 from this function. At world dendro I'm still seeing these being used by dplR users. It's a terrible stat. I'm taking it out. Help file amended. File: detrend.series.R and .Rd ---------------- - The function will now return the curves used for detrending the series if return.info is TRUE. Help file amended. - Added the Hugershoff curve as an method for detrending. It's done along the lines of ModNegExp with straight line if the nls call fails. - Added option to compute differences via subtraction rather than division. File: detrend.R ---------------- - See above. File: wavelet.plot.R ---------------- - Typos. * CHANGES IN dplR VERSION 1.6.8 - Note that Darwin Alexander Pucha Cofrep has been added as a developer to work on plotRings() etc. File: DESCRIPTION ---------------- - Fixing the version requirement for "animation" (>= 2.0-2) - Fixing the version requirement for "R.utils" (>= 1.32.1) - Introducing version requirement (>= 3.6) for suggested package "forecast" File: plotRings.R ---------------- - Big change made is to aspect so that the plot area is square: par(pty="s"). Will contact author to check on the reasonableness of this. Other changes. R CMD CHECK was throwing flags. Notes an issue in the length.unit arg to the function given differently in the Rd file vs the R file. Made a quick fix. Also made a small adjustment to the axis limits as it seemed there was a lot of extra white space in the plot. Removed bty for the legend. Adjusted par. Added documentation language. Not sure if I understand the full rationale behind the way the function is written though. There is a lot of redundant code. -AGB File: csv2rwl.R ---------------- Adding new function to read csv files in as rwl objects. Also adding that capability into read.rwl. Mikko should see if the error checks etc pass muster. File: read.crn.R ---------------- Adding argument 'long' (default is TRUE) for supporting (non-standard) long series with more than 4 characters used for the decade field. Use FALSE to revert to the old way of assuming 6 characters for site ID and 4 characters for decade. Thanks to Richard Telford for notcing the bug. File: read.rwl.R ---------------- Adding read2csv into read.rwl both as a format specification and as an auto detect. Mikko should see if the error checks etc pass muster. File: time.rwl.R ---------------- - Two convenience functions time.rwl and time.crn which extract the row.names of rwl and crn objects as numerics. Used as a S3 method with class rwl and class crn. This resulted in various changes to Rd files to make use of this capability. File: seg.plot.R ---------------- - Fixed an issue with ylim and yaxs which should cut out extra white space. File: xdate-dplR.Rnw ---------------- - Fixed typos * CHANGES IN dplR VERSION 1.6.7 File: latexify.R ---------------- - Fixed a bug in latexDate() which was causing failure on R-Devel. File: plotRings.R ---------------- - New function to plot a cross section. Contributed by Darwin Pucha-Cofrep and Jakob Wernicke File: DESCRIPTION ---------------- - Added in new authors (Darwin Pucha-Cofrep and Jakob Wernicke) - Added animation to Imports list - Added gmp to Suggests File: NAMESPACE ---------------- - Added plotRings to export and importFrom(animation, saveGIF) to support it * CHANGES IN dplR VERSION 1.6.6 File: helpers.R ---------------- - Internal helper fix.names() is more robust when working in the C locale. - Added internal helper function check.tempdir() - Added internal helper function requireVersion() File: latexify.R ---------------- - Fixed conversion bugs in the latexify() utility when working in the C locale. File: rasterPlot.R ---------------- - In rasterPlot(Cairo = TRUE, ...), added a version check for the Cairo package - Check if temporary directory is missing before use File: read.tucson.R ---------------- - Check if temporary directory is missing before use File: redfit.R ---------------- - Computation of the exact acceptance region of number of runs distributions in runcrit(), if not precomputed, depends on package "gmp" being installed (now optional). - Argument 'p' of redfit() and runcrit() may be numeric (which is the default) or "bigq", but the latter now fails if "gmp" is not installed. File: DESCRIPTION ---------------- - Version requirement for stringi bumped up to >= 0.2-3. Should have been done for dplR 1.6.1 - Version requirement for matrixStats bumped up to >= 0.50.2 because it's not possible to test 0.50.0, 0.50.1 with really old versions of R, e.g. 2.15.2. - Version requirement for png bumped up to >= 0.1-2: difficulties compiling 0.1-1 on a test system (type voidp not defined). Files: NAMESPACE, src/..., R/... ---------------- - Technical change: Native routines are registered. - Technical change: Visibility of native routines has been limited, following 6.15 "Controlling visibility" in R-exts. - Package "gmp" is now in Suggests, not in Imports. File: rcs.R ---------------- - Technical changes to facilitate the future implementation of signal-free rcs. File: src/redfit.c ---------------- - Fixed a pointer protection alert reported by rchk / Tomas Kalibera. Seemed like a false alarm, but the code is slightly cleaner as a result of the change. * CHANGES IN dplR VERSION 1.6.5 File: combine.rwl.R ---------------- -Changed so that combine.rwl returns class data.frame and class rwl. File: crn.plot.R ---------------- - Graphical parameters (xlim, ylim, ...) are handled better in crn.plot(). - Parameter 'ylim2' controls scale of secondary y axis. File: detrend and detrend.series.Rd ----------------- - Added Friedman under method argument in the help file. It was ommitted by mistake. - Bug in detrend fixed where make.plot was hard coded as FALSE which kept TRUE from being passed to detrend.series. File: helpers.R ----------------- - Added function find.internal.na() that reutrns the index of internal NA in a series. This will be used by the rwl.report() function and could be used (maybe) by the fill.internal.NA File: rwl.report.R ----------------- - Added function to provide a summary of information about an rwl object. This is still very much a work in progress. File: NAMESPACE ----------------- - Added rwl.report to export - Added rwl.report print method - Added import from plyr File: DESCRIPTION ----------------- - Added plyr (>= 1.8.3) to imports File: sea.R ----------------- - Note on commit from Zang: sea() simplified and computation of p-values fixed; computation of p-values and CI bands for sea() now based on ecdf() and quantile() functions. File: zzz.R ----------------- - .onUnload(): DLL / shared library is unloaded together with package * CHANGES IN dplR VERSION 1.6.4 File: DESCRIPTION ----------------- - Version number requirement for matrixStats is now (>= 0.50.0). File: chron-dplR-Rnw ------------------ - A new vignette for chronology building. This is pretty rough and could use some expanding. File: anos1.Rda ------------------ - The anos1 data from Christian Zang had been read into R incorrectly at some point. AGB noticed negative ring widths in the file and emailed Zang who sent along a corrected anos1.rda file. File: rasterPlot.R ------------------ - Added option 'Cairo' to control the preferred bitmap device. FALSE (the default) for png() as before, and TRUE for Cairo() as a new option. If the preferred device is unavailable, the other one will be tried. - rasterPlot() now reverts to normal plotting if bitmap device is unavailable or raster images are not supported. Previously an error would be produced. - The function now also works if no high level plot exists, i.e. if plot.new() has not been called. - Added option 'draw' to control whether the raster plot is to be drawn (TRUE, the default) or returned (FALSE) - png plot is initialized with plot.new() instead of the previous plot(type = "n", ...) arrangement. File: sea.R ----------- - Updated sea() to return bootstrapped CIs. File: treeMean.R ------------- - Made a new function to calculate the mean value of each tree when there are multiple cores. File: wavelet.plot.R -------------------- - wavelet.plot() now passes ... arguments to rasterPlot(). Files: detrend.R, detrend.series.R ---------------------------------- - New detrending method "Friedman" is Friedman's super smoother, implemented in stats::supsmu(). - When 'return.info' is TRUE, also the "f" parameter of method="Spline" is returned. File: DESCRIPTION ----------------- - New Suggested package: Cairo. Used (if available) in rasterPlot() if the png() device is unavailable or if the user specifies 'Cairo=TRUE'. File: NAMESPACE ------------- - Added treeMean function. - Importing dev.capture() from grDevices. Various .R files ---------------- - Argument values are checked more thoroughly: 'prec', 'header'. - 'if (condition) optionA else optionB' is used instead of 'ifelse(condition, optionA, optionB)' where appropriate. - Number of characters is used instead of display width where appropriate, or vice versa. Counterparts: strtrim() and substr(); nchar(type = "width") and nchar() with default 'type'; format(string, width = w, justify = "left") and stringr::str_pad(string, w, side="right"). - Using 'rows' and 'cols' arguments of matrixStats functions where appropriate. This should result in less time and memory being used. Thanks to Henrik Bengtsson for the tip and code examples. * CHANGES IN dplR VERSION 1.6.3 File: crn.plot.R ------------- - Added default x and y labels to plot. It would be better to use match.call() to check to see if xlab and ylab were passed in as dots but it is a very cumbersome bit of code for only a little difference to the user. File: spag.plot.R -------------- - Change to ylim in the plotting area. File: CITATION -------------- - Removed a piece of obsolete code which raised a NOTE in recent R-devel when checking --as-cran. File: DESCRIPTION ----------------- - A new field, MailingList, shows the address of the web interface to the dplR-help mailing list hosted on Google Groups. - New Imported packages: Matrix, matrixStats, R.utils. - New Suggested package: testthat. Unit tests are now done with testthat instead of RUnit. - RUnit is no longer Suggested. - Updated version requirement: gmp (>= 0.5-5). Earlier versions of gmp do not export is.bigq() which is required by dplR. The previous version requirement (>= 0.5-2) was a mistake. - Updated version requirement: R (>= 2.15.2). For some reason, R CMD check does not play nice with testthat on earlier versions of R. - Removed one of the URLs in the URL field. BDR sent and email saying that having two URLs was breaking R and that we should put a comma in betwixt them. But the comma is problematic as it is a valid part of some URLs. We decided the better part of valor was to keep only one URL and not fight this particular battle. File: NAMESPACE --------------- - Importing captureOutput() from R.utils. - Importing package Matrix. - Importing various functions from matrixStats. - Importing packageDescription() from utils, dropped installed.packages(). - Exporting net(). - Added previously missing imports. From grDevices: dev.capabilities(), dev.cur(), png(), dev.off(), dev.set(), devAskNewPage(). From utils: read.table(), write.csv(). As these packages are attached by default, the missing imports would not have been a problem in most cases. Various .R files ---------------- - Using deparse.level=0 in some calls to cbind() and rbind() when names are not needed in the result. Due to this change, the first column of the $bins matrix in the return value of ccf.series.rwl(), corr.rwl.seg(), and corr.series.seg() does not have (undocumented) column names anymore. - Using base::rowSums and functions from the matrixStats package to speed up some operations on rows or columns of matrices. - Reduced the number of calls to options() by one when setting and restoring options - Argument values, particularly those that should be character vectors, are checked more thoroughly. This makes many functions more robust against unusual values: wrong type, "bytes" encoding, zero length or NA. Some values that previously failed are now silently accepted by coercion to character, extraction of first element when a single string is expected, and / or interpretation of a zero length argument as an empty string. NA is equivalent to "NA" in detrend.series() and skel.plot() where it is used for plotting or text output, but forbidden otherwise, e.g. in chron(). File: ffcsaps.R --------------- - Increased performance by using sparse matrices from the Matrix package. File: helpers.R --------------- - Internal function vecMatched does not care if nzchar() starts returning NA some day. File: latexify.R ---------------- - Improved efficiency when handling a large number of strings with the "bytes" Encoding. The code now uses R.utils::captureOutput() instead of a home-baked re-implementation of (parts of) utils::capture.output(), which is also a more compact solution in terms of lines of code in dplR. See Henrik Bengtsson's notes at http://www.jottr.org/2014/05/captureOutput.html (referenced on 2015-01-07). - Removed an unused variable. File: net.R ----------- - New function for computing the NET parameter (Esper et al., 2001). File: read.fh.R --------------- - read.fh() now ignores empty lines at the end of series. File: read.tucson.R ------------------- - New argument 'edge.zeros', TRUE (default) or FALSE. If TRUE, possible leading and trailing zero values in tree-ring series are kept. If FALSE, such values are dropped. The latter is how the function has worked between (pre-)release 1.5.5 and the previous release, and may be considered a bug or a misfeature. To reiterate, the default behavior has changed in some cases. The alternative, 1.5.5--1.6.2 behavior can be restored by changing the value of the argument. Reported by Daniel Bishop and Neil Pederson. File: redfit.R -------------- - Fixed a bug where redfit() would fail if the "stats" package was not attached. File: series.rwl.plot.R ----------------------- - Worked around a problem in graphics::abline(): the function fails when a linear model is given and the "stats" package is not attached. The solution is to extract the coefficients of the model and use those in the call. File: write.crn.R ----------------- - Better coding style: avoid using assign(). File: write.tridas.R -------------------- - Implementation detail: When checking the package version, packageDescription() is now used instead of installed.packages(). * CHANGES IN dplR VERSION 1.6.2 No functional changes. A unit test was changed so it would not fail on the solaris-sparc CRAN platform. Also in some other tests, identicality checks were replaced with near equality checks. * CHANGES IN dplR VERSION 1.6.1 File: glk.R and glk.Rd ------------- - Modified by Christian Zang in response to a bug report by Allan Buras. In the case of no change from year i to year i+1 in both series then the glk sign will for both be 0 and the sum of both is then also 0 and will not be accounted for correctly in the sum of synchronous years. Zang and Buras have patched and Zang updated the help file to reflect the change as: "This implementation improves the original formulation inasmuch as the case of neighbouring identical measurements in the same years is accounted for. Here, it is treated as full agreement, in contrast to only partial agreement in the original formulation." File: crn.plot.R ------------- - Removed automatic plotting of chronology name in main title. File: plot.wavelet.R ------------- - Fixed small bug in ylim for the chronology in the plot. File: NAMESPACE --------------- - Added rasterPlot, latexify() and latexDate() to export list - Import readPNG from png. - Import more functions from grid. - Import stri_trans_nfc from stringi. File: DESCRIPTION ----------------- - New Suggested packages: Biobase, dichromat, knitr, tikzDevice, RColorBrewer. Of these, dichromat, knitr, and tikzDevice are for document building (see math-dplR.pdf below). Biobase is for making access to math-dplR.pdf easier with openPDF(). RColorBrewer provides an alternative palette to an example in wavelet.plot.Rd. - New Imported packages: png and stringi. File: chron.R ------------- - Added pass-through of arguments from chron() to ar() Files: corr.rwl.seg.R, corr.series.seg.R, interseries.cor.R, rwi.stats.running.R, xskel.ccf.plot.R, xskel.plot.R ------------------------------------------------------------ - Parameters not changed by assignment anymore (small technical detail) File: common.interval.R ----------------------- - Bug fix: make.plot=TRUE threw an error when input data.frame had leading or trailing all-NA rows File: corr.rwl.seg.R, skel.plot.R --------------------------------- - Performance optimization, including the use of dev.hold() and dev.flush() File: corr.series.seg.R ----------------------- - Bug fix: Argument 'method' was not used for moving correlations. Instead, the method used was always "spearman". File: detrend.R --------------- - Updated .export of foreach() to match previous changes to the loop body File: latexify.R ---------------- - New utility functions latexify() and latexDate() for use in vignettes New files in subdirectory inst/doc: ----------------------------------- - math-dplR.pdf is a vignette-ish document about the mathematical details of some dplR functions (room for expansion in the future). It is packaged as a static PDF due to the long build time. However, the source is available as required by the CRAN policies for an open source package. Arguably the staticness also makes sense in light of the document presenting non-changing information about the package. If the functions analyzed in the document change fundamentally (which is not to be expected), the relevant parts of the document must be rewritten and the document recompiled. Compare this to the idea that regular vignettes usually include concrete examples about how the package can be used. In that case it makes sense to compile the document often to verify that the examples work. - math-dplR.Rnw.txt is the source of math-dplR.pdf, with extra file extension .txt to prevent automatic rebuild by R CMD build and R CMD check - math-dplR.bib is the BibTeX bibliography of the document - math-dplR.R is the R source extracted from the document - build-math-dplR.R is a build script File: morlet.R -------------- - Added some input checks File: powt.R ------------ - Handle cases where the running mean of a series is 0 New file rasterPlot.R --------------------- - New function rasterPlot(). Adds a raster image drawn with low level graphics commands to the current high level plot. Files: rcompact.c, readloop.c ----------------------------- - Made the C code of read.compact() and read.tucson() more resilient to unusual inputs File: redfit.c -------------- - Avoid using the long double type in some cases. Hopefully this will speed up otherwise unbearable computation times on some systems. Files: series.rwl.plot.R ------------------------ - Fixed clipping of text in the lower right corner of the plot File: skel.plot.R ----------------- - Adjusted alignment, size and clipping properties of viewports. Now a plot with less than the full number of rows can fit in a smaller device and text on the sides won't be clipped. File: spag.plot.R ----------------- - Added two options to spag.plot(). 'useRaster': draw the tree-ring series as a raster image? (default 'FALSE') 'res': resolution of the tree-ring series when 'useRaster' is 'TRUE' File: timeseries-dplR.Rnw ------------------------- - wavelet.plot(useRaster=TRUE) in the vignette reduces the size of both timeseries-dplR.pdf and the package tarball File: wavelet.plot.R -------------------- - Added three options to wavelet.plot(). 'useRaster': draw the filled contours as a raster image? (default 'FALSE') 'res': resolution of the filled contours when 'useRaster' is 'TRUE' 'reverse.y': if TRUE, Y-axis of the filled contour plot is reversed - A subtle change to the default value of wavelet.levels: To get the sequence 0, 0.1, 0.2, ..., 1 it is best to use (0:10)/10 instead of seq(from=0, to=1, by=0.1). Parentheses in the former are used for clarity of meaning. - Added some input checks - Small optimizations - More graphical parameters are restored after the function has run - Better reuse of code between side.by.side=TRUE and side.by.side=FALSE - Use of dev.hold() and dev.flush() Files: xskel.ccf.plot.R and xskel.plot.R ---------------------------------------- - Code optimizations - Small changes to output * CHANGES IN dplR VERSION 1.6.0 File: TODO ------------------------- - Added a TODO list that follows the todo.ell format. File: NAMESPACE ------------------------- - Added chron.plot to export list. - Added interseries.cor to export list. - Added plot.rwl as an S3Method. - Added plot.crn as an S3Method. - Added summary.rwl as an S3Method. - Added insert and delete.ring functions. File: xskel.ccf.plot.R and xskel.plot.R --------------- - New plotting functions to help crossdate with skeleton plot and cross correlation plots. File: ccf.series.rwl.R --------------- - Switched the order of x and y in the call to ccf(). This makes a great deal more logical sense now as a missing ring shows up with a positive lag rather than a negative lag. - Changed color scheme a bit to look less harsh Files: ccf.series.rwl.R, corr.series.seg.R, series.rwl.plot.R ------------------------------------------------------------- - New convenience feature: if the length of 'series' is 1, it is interpreted as a column index to 'rwl', and the corresponding series is left out of the master chronology. File: ffcsaps.R --------------- - Small optimization: using "usually slightly faster" 'tcrossprod(x)' instead of 'x %*% t(x)' File: insert.ring.R ------------------------- - insert.ring and delete.ring functions for editing rw vectors. Very simple. File: crn.plot.R ------------------------- - Added several new plotting options to give users more control of plot - Aliased crn.plot to plot.crn so it can be used as an S3 plot method. Thus a user can now to bar <- chon(foo); plot(bar) - Help revised considerably File: read.crn.R ------------------------- - Added class "crn" to output object. File: cana157.rda ------------------------- - Added class "crn" to object. File: detrend.R and detrend.series.R ------------ - Added an Ar detrend method. Revised plotting in detrend.series - Added a verbose option to print out parameters used in detrending - Added a return.info option to write a list of parameters used in detrending File: powt.R ------------ - Originally, the transformed series were rescaled to their original mean and variance, which can lead to negative values for the (supposedly) raw tree-ring data. This is not necessary, and has been removed now. File: rwl.stats ------------------------- - Added an S3 summary method for rwl objects so that summary(foo.rwl) calls rwl.stats(foo.rwl) Folder: vignettes ------------------------- - Added a vignettes folder File: dplR.sty ------------------------- - Very basic sty file File: dplR.bib ------------------------- - Refs for vignettes File: intro-dplR.Rnw ------------------------- - A vignette to introduce dplR File: xdate-dplR.Rnw ------------------------- - A vignette for cross-dating File: timeseries-dplR.Rnw ------------------------- - A vignette for time series stuff. Very basic! File: corr.series.seg.R -------------------- - Added method argument to specify method for cor.test(). Defaults to "spearman." File: corr.rwl.seg.R -------------------- - Removed yr.range() function in favor of yr.range() in helpers.R. They are identical for all practical purposes. - Added method argument to specify method for cor.test(). Defaults to "spearman." File: interseries.cor.R ------------------------- - New function interseries.cor. File: read.compact.R ------------------------- - Added class "rwl" to output object. File: read.fh.R ------------------------- - Added class "rwl" to output object. File: read.tridas.R ------------------------- - Added class "rwl" to output object $results. File: read.tucson.R ------------------------- - Added class "rwl" to output object. File: anos1.rda ------------------------- - Added class "rwl" to object. File: ca533.rda ------------------------- - Added class "rwl" to object. File: co021.rda ------------------------- - Added class "rwl" to object. File: gp.rwl.rda ------------------------- - Added class "rwl" to object. File: rwl.plot.R ------------------------- - New wrapper to plot rwl objects. File: spag.plot.R ------------------------- - Cosmetic changes to plot. File: seg.plot.R ------------------------- - Cosmetic changes to plot. File: rwi.stats.running.R ------------------------- - Added signal-to-noise ratio as an output. Followed pg 109 in Cook's chapter in Hughes et al. 2011. - New outputs 'n.cores' and 'n.trees' show the total number of cores and trees in the window, respectively. At least one non-missing value is required for a core and tree to be counted. - When using period = "common" in rwi.stats() or rwi.stats.running(), the number of trees 'n' in the return value is now 0 instead of the full number in case there are no complete cases in the running window. - 'c.eff' in the return value is now 0 if no correlations were computed - Added a method argument to change the type of correlation method performed. - Optimizations Files: write.compact.R, write.crn.R, write.rwl.R, write.tridas.R, write.tucson.R ------------------------------------------- - The write.* functions now return the name of the output file. Previously it was documented that there was no return value. - Examples in the corresponding .Rd files were modified to use tempfile()s instead of writing to the working directory, which was potentially harmful. * CHANGES IN dplR VERSION 1.5.9 Files: dplR.h, rcompact.c, redfit.c ----------------------------------- - Restructured header inclusions - dplR.h declares new function dplRlength (internal use) File: dplR.c ------------ - New file for general C functions internally used by dplR Files: exactmean.c, exactmean.R, gini.c, gini.coef.R, readloop.c, read.tucson.R, sens.c, sens1.R, sens2.R, tbrm.c, tbrm.R ------------------------------------------------------------------- - Use .Call() instead of .C() for interfacing with C code - The C functions need less support (arguments, argument checking) from the calling R function Files: exactsum.c, exactsum.h ----------------------------- - Reuse of code between functions by using the C preprocessor - New (internal to dplR C code) function cumsum (cumulative sum, overwrites input array) - size_t n File: gini.c ------------ - Simplified expression for gini coefficient reduces number of arithmetic operations performed - Fewer calls to other functions (use of the new cumulative sum function) File: readloop.c ---------------- - Fixed buggy handling of malformed Tucson format series that have no data and no stop marker. Previously, this hypothetical case could have caused a read from a location just outside the array bounds. However, in most cases that would probably not have been a problem. File: rwi.stats.running.R ------------------------- - Bug fix: when using period = "common" in rwi.stats() or rwi.stats.running(), the number of trees 'n' used in EPS and shown in the return value is now the total number of trees in the data.frame, taking into account the fact that rows with any missing values are effectively dropped. Thanks to Donald Zhao for reporting. * CHANGES IN dplR VERSION 1.5.8 File: tbrm.Rd --------------- - Improved (slightly) the trbm() examples. Files: exactmean.R, gini.coef.R, sens1.R, sens2.R, tbrm.R --------------------------------------------------------- - Changed calls to .C (where we use DUP = FALSE) so that the binding of NaN will not change, fixing multiple instances of a bug in dplR code revealed by changes in recent R development versions. We were accidentally breaking CRAN's "malicious or anti-social" policy. Thanks to Professor Brian D. Ripley for reporting. Now R CMD check --as-cran passes fine. File: DESCRIPTION - Added Jacob Cecile as a contributor File: detrend.R --------------- - Adjusted for the changes in detrend.series(), i.e. added the argument constrain.modnegexp. File: detrend.series.R ---------------------- - Fixed a bug where RWI could go negative. Thanks to Jacob Cecile for reporting the bug and contributing a proposed solution. - A new argument: constrain.modnegexp. It is now possible to constrain the modified negative exponential function to non-negative values at infinity. - Warn if fit is not all positive (backup methods will be used) - Use simpler but equivalent rules for checking validity of ModNegExp fit - Small optimizations (rep.int; save and reuse length of input) File: redfit.c -------------- - Avoid possibly buggy behavior in the unlikely case that cbind() or length() have been redefined by the user File: redfit.R -------------- - Use slightly faster .rowSums() instead of rowSums() - Simplified arithmetic expressions in getdof(): no multiplying by 2 - Precomputed squared numbers in getdof() - Fixed Welch, Hanning, Triangular and Blackman-Harris windows to be DFT-even - Computed more precise values for the 6 dB bandwidths of each window, also for short windows. Uniform sampling was assumed. - Two internal functions moved to top level, previously inside print.redfit() File: tbrm.R ------------ - Check that C has length 1 * CHANGES IN dplR VERSION 1.5.7 File: DESCRIPTION ----------------- - Import gmp (>= 0.5-2) File: NAMESPACE --------------- - New imports from gmp and utils - Export redfit() and runcrit() Various .R files ---------------- - Check that length of vector does not overflow integer datatype before use of .C() - Avoid possible name clashes when using foreach with parallel backends File: common.interval.R ----------------------- - Optimizations (for example, less subsetting of the 'rwl' data.frame) - Better handling of corner cases (zero dimensions etc.) - In the plot (make.plot = TRUE), length of lines was adjusted: First year a, last year b is 'b - a + 1' years, not 'b - a' years File: corr.rwl.seg.R -------------------- - New feature: allow the master series to be built from a second set of tree ring series by using a data.frame 'master' argument - Replaced some for loops with cleaner vectorized operations or apply(). File: helpers.R --------------- - Fixed a bug in fix.names(), related to creating unique short names. The bug affected read.tridas(), write.compact(), write.tridas() and write.tucson() but probably manifested itself quite rarely. File: rwi.stats.running.R ------------------------- - Speedup by using rep.int() instead of rep() File: sea.R ----------- - Extra input checks (e.g. x must have explicit, non-automatic row-names) - Some matrices now have the correct type (numeric instead of logical) right from the beginning - Small optimization: a temporary matrix is completely overwritten on every round of a loop, so no need to reinitialize - Braces always used in if (else) constructs Files: redfit.R, redfit.c ------------------------- - New function redfit() based on REDFIT by Schulz and Mudelsee. Also another exported function runcrit(). * CHANGES IN dplR VERSION 1.5.6 File: write.tucson.R ------------------------ - Changed series IDs to justify left instead of right. I'm not sure why they ever wanted to be justified left. Silly. (AGB) File: NAMESPACE --------------- - Exporting new function common.interval() File: common.interval.R ------------------------ - New function common.interval() trims a rwl object to a common interval using one of three methods. Contributed by a user Filipe Campelo (fcampelo@ci.uc.pt). This is his first contribution. Added to author list in DESCRIPTION. File: corr.rwl.seg.R -------------------- - Bug fix: series names were not shown (numbers were shown instead) - Bug fix: there were off-by-one errors in the length of the bars File: DESCRIPTION ----------------- - Changed author and maintainer to Andy Bunn from Andrew G. Bunn to keep parity between the names and the email address AGB uses to submit to CRAN. This was made at the request of Kurt Hornik at CRAN * CHANGES IN dplR VERSION 1.5.5 File: NAMESPACE --------------- - Exporting new functions Various .R files: ----------------- - Use 'nzchar(x)' instead of 'nchar(x) > 0' File: rwi.stats.running.R ------------------------- - Added prewhitening option to rwi.stats.running() and by extension rwi.stats(). There are two new arguments prewhiten and n that are passed to normalize1() as in the xdating functions e.g., corr.rwl.seg(). Help file changed. Files: corr.rwl.seg.R, seg.plot.R, rwl.stats.R, spag.plot.R ----------------------------------------------------------- - Support for input of length 1 File: corr.rwl.seg.R -------------------- - Fixed 'ylim' in plot() - Fixed "no guides" case - stops with a clear error message if 'rwl' has too few rows for the given 'seg.length' and 'bin.floor' combination File: fill.internal.NA.R ------------------------ - New function fill.internal.NA() fills NA values internal to a series. Written by Andy Bunn and Mikko Korpela. Help page added as well. File: pointer.R --------------- - New function pointer() calculates pointer years from a group of ring-width series. Written by Pierre Mérian, adapted for dplR and improved by Andy Bunn and Mikko Korpela. Help page added as well. File: rcs.R ----------- - Graceful handling of empty input File: read.compact.R -------------------- - Pretty printing of summary output, no more ragged lines File: read.fh.R --------------- - Pretty printing of summary output, no more ragged lines - More robust detection of block and single column data representations - Data block, when using block representation, is interpreted as fixed width fields (10*6)). Reference: TRiCYCLE Users Manual, Version 0.2.6. - Different units are supported. Default is 1/100 mm. - Each data block is mapped to the correct header block. Previously, there was a risk of using the wrong header if the file contained data in formats other than "Tree" or "Single". Also, the end position of any data block could be off if data with different formats was present in the file. Presumably the function would have failed with an obscure error message. - Added support for site, tree, core, etc. metadata. Results are given as an attribute of the return value, named "ids". - Added support for MissingRingsBefore (pith offset) metadata. Results are given as an attribute named "po". File: read.tucson.R ------------------- - Fixed trimming of all-NA rows - Fixed a bug that could crash R if the fixed-width columns of the input file did not follow the (loose) specifications of the Tucson format - AB: Note to dplR developers that this is a result of a poor standard in the Tucson format but this fix is needed to work with files that are on the ITRDB. Interestingly, dpl and ARSTAN are more robust to these kinds of inconsistencies. Mikko's note: always check that your assumptions a piece of input are correct before using the said input to compute array indices, particularly in C code. - Can deal with CR CR LF newlines by reading the whole file into memory at first and stripping empty lines - Can read non-standard files where one or more of the stop markers is the 11th data column of its row - Can read non-standard files where columns don't have their proper widths, including tab-delimited files - Can read non-standard files where missing data is marked with non-numeric characters - Printed summary is justified, no more ragged lines - Interprets lines containing "#" characters (in positions 1-78) as comments. For now, comments are ignored. - Fixed a bug that could cause mixing of values from two or more measurement series sharing the same ID. Now it is an error if the input file contains more than one measurement for any year, ID pair. - Accommodate mid-series upper and lower case differences: If a series does not end with a stop marker, see if the series ID of the next row after the last belonging to the series without a stop marker matches when case differences are ignored. If so, interpret these as the same series. File: write.tucson.R -------------------- - Instead of always using 1000, 999 is now randomly converted to either 998 or 1000 (prec == 0.01) => no bias (even if small) * CHANGES IN dplR VERSION 1.5.4 File: DESCRIPTION ----------------- - Depends: R (>= 2.15.0) to accommodate use of .filled.contour() in wavelet.plot(). Also enables the use of paste0(). File: NAMESPACE --------------- - Imports from package stringr (also in DESCRIPTION) - Exports new function autoread.ids() Various .R files ---------------- - use paste0(...) instead of paste(..., sep="") - prettier, more consistent formatting of source code File: helpers.R --------------- - New internal function check.flags() requires that its arguments are TRUE or FALSE File: read.ids.R ---------------- - Optional automatic detection of the site-tree-core scheme (stc="auto") - Optional fixing of typos in series names - Optional (adaptive) case insensitivity - New wrapper function autoread.ids() calls read.ids with stc="auto" and an alternative set of parameter values File: read.tucson.R ------------------- - More robust detection of header File: wavelet.plot.R -------------------- - Switched from using an .Internal() call to using new bare-bones function .filled.contour() for the plotting. This is at the request of Prof. Ripley who wrote that 'Packages should not call .Internal(): it is not part of the API, for use only by R itself and subject to change without notice.' In 1.5.3, the use of .Internal() or .filled.contour() was an if-else decision based on the version of R, but now the latter function is always used, making R >= 2.15.0 required. - enabled translation of default value of 'key.lab' - checks 'wave.list$x' and 'wave.list$period' Files: write.compact.R, write.tucson.R -------------------------------------- - Useless uses of eval() removed 2012-03-05 Andy Bunn * CHANGES IN dplR VERSION 1.5.3 File: CITATION -------------- - Uses the new bibentry() style - Has an automatic entry for the dplR manual (R >= 2.14.0 requirement) - Entry for "Statistical and visual crossdating in R using the dplR library" was updated File: DESCRIPTION ----------------- - Encoding: UTF-8 - Depends: R (>= 2.14.0) - Author and Maintainer fields dropped (made obsolete by Authors@R) File: NAMESPACE --------------- - import() or importFrom() from all the "base" Priority packages that are used in dplR. Previously, the use of functions from those packages seems to have relied on the assumption of them being attached. Quote from "Writing R Extensions": "Packages implicitly import the base namespace. Variables exported from other packages with namespaces need to be imported explicitly using the directives import and importFrom." Clearly, packages with Priority "base" (different from the package / namespace called "base") fall into the category of "other packages". - importFrom() used in more cases - exports rwi.stats.legacy() File: powt.R ------------ - New function for power transformation after Cook and Peters File: rcs.R ----------- - Allows for standardization by subtraction Files: read.crn.R, read.tucson.R -------------------------------- - Fix handling of (effectively) empty lines (thanks to Heather Gamper for reporting) File: rwi.stats.R ----------------- - rwi.stats() was renamed to rwi.stats.legacy(), potentially useful for comparing the results of the old and new code File: rwi.stats.running.R ------------------------- - zero.is.missing now has default value TRUE - rwi.stats() is now a wrapper to rwi.stats.running() - allows 'ids' to have extra rows if all names of 'rwi' appear as its row names File: strip.rwl.R ----------------- - New function for EPS-based chronology stripping File: wavelet.plot.R -------------------- - Replaces .Internal(filledcontour) with .filled.contour in R >= 2.15.0 2012-01-19 Mikko Korpela * CHANGES IN dplR VERSION 1.5.2 - Requires R >= 2.12.0 (use of markup in some Rd \title{} sections). - Documentation has been cleaned up / uses better markup. File: ffcsaps.R --------------- - Checks that 'x' and 'y' are coercible to _numeric_ vectors File: read.ids.R ---------------- - Checks that 'stc' contains nothing but integral values and has length 3 Files: write.compact.R, write.tridas.R, write.tucson.R ------------------------------------------------------ - Accept unknown arguments ('...') 2011-12-19 Mikko Korpela * CHANGES IN dplR VERSION 1.5.1 File: corr.rwl.seg.R -------------------- - New parameters 'master' and 'master.yrs': Instead of letting corr.rwl.seg() compute master series based on 'rwl', the user can use her own master series. - Ensures that rwl (and master) have consecutive years in increasing order - Uses full form "greater" instead of "g" in calls to cor.test() File: corr.series.seg.R ----------------------- - Uses full form "greater" instead of "g" in calls to cor.test() 2011-11-23 Mikko Korpela * CHANGES IN dplR VERSION 1.5.0 Various .R files: ----------------- - Use TRUE instead of T - sapply() replaced with vapply() or vectorized operations File: detrend.series.R ---------------------- - Checks that there are no NAs in the middle of the series. Series from dplR rwl data.frames don't have mid-series NAs (unless manipulated by the user), but other data might. File: read.crn.R ---------------- - Calls to read.fwf() now set the colClasses parameter. This gives more predictable behavior when the input file contains non-integer data where integers are expected. That is,.the function stops with a clear error message, whereas previously, the resulting data.frame would have contained seemingly random zeros. - Some optimizations (vectorization, less copying, etc.) - In the non-standard situation of multiple series per file, the previous versions assumed the same range of years for all series. This rule, breaking of which would give strange results, no longer applies. File: read.fh.R --------------- - Replaced read.csv() with readLines() - Unnecessary captures removed from regular expressions - Small optimizations (e.g., positions() was replaced with a more efficient solutions) - The possibly dangerous removal of zeros, even from the middle of series, was rewritten - Gives a clear error message when a data series has unexpected length - Handles empty files / files with zero records better File: read.tucson.R ------------------- - Calls to read.fwf() now set the colClasses parameter (see explanation above) File: rwi.stats.running.R ------------------------- - Fixes a bug where the function would not work if 'ids' was NULL 2011-11-14 Mikko Korpela * CHANGES IN dplR VERSION 1.4.9 File: rcompact.c ---------------- - read.compact() now accepts series IDs consisting of any sequence of printable ASCII characters. 2011-11-06 Mikko Korpela * CHANGES IN dplR VERSION 1.4.8 File: NAMESPACE --------------- - Made some internal functions truly internal by removing them from the export list File: dplR-internal.Rd ---------------------- - The file was removed Various .c and .h files: ------------------------ - NULL, TRUE and FALSE are used for clarity - Rboolean type is now used more extensively for truth values - stddef.h is #included where NULL is used Various .R files: ----------------- - When indexing data.frames, use df[[foo]][bar] instead of df[bar, foo] when foo is a single index (or df$foo[bar] instead of df[bar, "foo"]). The former is supposed to be faster than the latter. https://stat.ethz.ch/pipermail/r-devel/2011-October/062313.html - Some input checks added File: combine.rwl.R ------------------- - Avoids some conversions between matrix and data.frame File: ffcsaps.R --------------- - For loop removed in ffppual (constant number of iterations) - Useless instances of cbind() and rbind() removed - Avoids computing or passing as arguments things that are already known or constant - ffsorted2() is a modified version of ffsorted() which should speed things up a little by making a rev() call unnecessary - Unnecessary call to pmax() removed - Added some input checks - order() is now used instead of sort(method="shell"), because the requirements for the latter being a stable sort are unclear (?sort in R 2.13.2), and stability of the sort is required in some parts of ffcsaps (may be desirable in others) File: normalize1.R ------------------ - For consistency, the first part of the returned list is now a matrix regardless of the value of 'prewhiten'. This also gives an amazing speed improvement in corr.rwl.seg: from 42 seconds to 1 second in the ?corr.rwl.seg Example, modified with prewhiten=FALSE, make.plot=FALSE. This corrects the performance degradation introduced in dplR 1.2.7. Tested on a computer with a Core 2 processor, 3.0 GHz. File: rcompact.c ---------------- - Now accepts series IDs with spaces. File: seg.plot.R ---------------- - Uses order() and numeric indexing instead of sort() and indexing with names File: skel.plot.R ----------------- - Replaced one for loop with vectorized operations File: spag.plot.R ---------------- - Uses order() and numeric indexing instead of sort() and indexing with names 2011-09-14 Mikko Korpela * CHANGES IN dplR VERSION 1.4.7 Various .R files: ----------------- - For data.frames, row.names() is now used instead of rownames(). The opposite change was applied in one location (not a data.frame). Also, names() is used instead of colnames() where appropriate. This is because names() and row.names() are preferred to colnames() and rownames(), respectively, when dealing with data.frames. File: read.ids.R ---------------- - Fixed a bug introduced in dplR 1.4.1 where the function would not work for non-numeric identifiers or when identifiers did not fall in the range from 1 to n. - Now respects input identifiers where possible: If all substrings denoting tree or core are integers, they are used. Otherwise, sorted unique identifiers (number of which is n.unique) are mapped to numbers 1:n.unique. - Now allows the sum of the site-tree-core mask to be smaller than 8. The remaining characters will be ignored. This can be handy if there are additional levels in the ID hierarchy. Then, the series with matching ID strings up to the sum of the site-tree-core mask will be given matching tree and core numbers. File: rwi.stats.running.R ------------------------- - Now averages data from series with the same tree/core ID combination before computing any statistics. - Now has the option to treat zeros as missing data (parameter zero.is.missing). Defaults to FALSE which gives identical results compared to previous versions of the function, but should probably be set to TRUE for most purposes. 2011-09-01 Mikko Korpela * CHANGES IN dplR VERSION 1.4.6 File: corr.rwl.seg.R -------------------- - Fixed a bug in the $flags part of the list returned by the function. Previously, a single flagged segment / series was not reported. File: corr.series.seg.R ----------------------- - Fixed a bug where moving correlations were computed using a window one year longer than seg.length. This problem had gone unnoticed when fixing some related bugs for version 1.4.5. - Changed the x and y limits of the plot. Now the computation of the y limits does not use p-values as an input (the y-axis is correlation, not p-value). x limits are now set explicitly, and generally speaking there is more space around the plotted data. 2011-08-11 Mikko Korpela * CHANGES IN dplR VERSION 1.4.5 Various .R files: ----------------- - Updates to sequence generation, e.g. use seq_along and seq_len where applicable Files: ccf.series.rwl.R, corr.rwl.seg.R, corr.series.seg.R, series.rwl.plot.R ----------------------------------------------------------------------------- A bunch of related bugs were fixed, some not present or already (partially) fixed in some files: - Fixed a bug where segment length was always actually one year longer than the specified seg.length (not in corr.rwl.seg.R, corr.series.seg.R). - Fixed a bug where the requirement of fitting at least two segments did not take into account overlapping segments (seg.lag), therefore requiring more than the minimum number of years (not in corr.rwl.seg.R). Note: qa.xdate() still requires nrow(rwl) >= seg.length*2. - Added a "plus 1" option (floor.plus1) to location of first segment. Works together with parameter bin.floor. Default value is FALSE. - In corr.rwl.seg.R, cleaned up the code and fixed some bugs related to segment boundaries. - In corr.rwl.seg.R, fixed a bug where multiple disconnected red segments were not drawn correctly. - In corr.rwl.seg.R, removed unused variables / operations - In corr.rwl.seg.R and corr.series.seg.R, arguably prettier x axis ticks and labels are used (now separated by seg.length). - In corr.series.seg.R, an additional error check was added File: tbrm.c ------------ - "Nothing to sum" case, when there are some numbers but all are too far from median, returns NaN instead of NA. Finally, this is consistent with tbrm() coded in R, prior to dplR 1.2.7. File: write.compact.R --------------------- - Fixed two precedence issues that luckily didn't result in erroneous behaviour of the function 2011-07-03 Mikko Korpela * CHANGES IN dplR VERSION 1.4.4 * Changes in documentation will not be reported anymore Various .R files: ----------------- - Parameters not changed by assignment anymore (complaint by codetools). Applies to direct assignment; assign() etc go unnoticed by codetools. File: ccf.series.rwl.R ---------------------- - Cosmetic changes (white spaces, assignment operator) - Parameter 'cex' is given in the call to xyplot, like 'col.line'. Both go to 'panel' in '...'. - Parameter 'col' removed from function 'panel' - Some extra steps are taken because we want to ensure correct operation now when formal parameters are not changed by assignment - (1 + 1 - pcrit) / 2 == 1 - pcrit / 2 (at least within floating point precision), so we use the latter which is a simpler form File: chron.R ------------- - Argument checking updated File: corr.series.seg.R ----------------------- - Some extra steps are taken like in ccf.series.rwl.R - (1 + 1 - pcrit) / 2 == 1 - pcrit / 2 File: detrend.R --------------- - Parameter 'f' now has an explicit default value File: detrend.series.R ---------------------- - Unnecessary double checking of 'method' argument removed - Parameters 'f' and 'y.name' now have explicit default values - Graphical parameters are reset on.exit() File: ffcsaps.R --------------- - Uses complete parameter names in calls to sort File: glk.R ----------- - Large parts rewritten - 'glk(ca533)' example in glk.Rd runs about 100 times faster - Now explicitly requires that the non-NA overlap between series be contiguous, which should be true in dplR. This was not checked in previous versions. File: hanning.R --------------- - Slightly more efficient computation File: i.detrend.R ----------------- - Parameter 'f' now has an explicit default value File: i.detrend.series.R ------------------------ - Parameter 'f' now has an explicit default value - Fixed a bug where the values of the parameters 'f', 'nyrs' and 'pos.slope' were not reflected in the returned result, only in the picture. - Saved result is used instead of detrending twice (once with all methods, then with selected method) - "Enter a number" is asked until a valid number is received. Normal interrupt sequences work. File: morlet.R -------------- - Removed 'param' and made 'k0=6' an explicit default in morlet.func() - seq(from=1, to=n) - 1 == seq(from=0, to=n-1) - log2(x) instead of log(x) / log(2) - Cosmetic changes File: series.rwl.plot.R ----------------------- - Some extra steps are taken like in ccf.series.rwl.R - Graphical parameters are reset on.exit() File: wavelet.plot.R -------------------- - log2(x) instead of log(x) / log(2) File: write.tridas.R -------------------- - Fixed a bug in the handling of parameter 'crn.units' 2011-07-01 Mikko Korpela * CHANGES IN dplR VERSION 1.4.3 * Changes below by Mikko Korpela Directory: po ------------- - A new directory used for language translations http://cran.r-project.org/doc/manuals/R-exts.html#Internationalization Directory: inst/po ------------------ - Location for compiled translations Directory: inst/po/fi and contents --------------------- - Compiled Finnish translations Various .R files: ----------------- - Diagnostic and normal output messages were edited to facilitate translations. Messages consist of logical units (no small fragments, but sequences of sentences is possible), and gettext() and gettextf() are used. Also plot labels, titles and legends are ready for translation File: po/R-dplR.pot ------------------- - A translation template for messages in R code File: po/dplR.pot ------------------- - A translation template for messages in C code File: po/R-fi.po ---------------- - Finnish translations of messages in R code File: po/fi.po -------------- - Finnish translations of messages in C code File: src/dplR.h ---------------- - A new file that (initially) contains definitions used for looking up translations of messages appearing in C code File: src/rcompact.c -------------------- - Supports translation of messages via _(...), where ... is the string to translate File: DESCRIPTION ----------------- - Added a link to the R-Forge dplR development page File: cms.R ----------- - Better checking of pith offset names - Big performance improvement, mostly due to using vector operations instead of a for loop - Results slightly different (on the order of 1e-15) due to polyroot having been replaced with quadratic formula, order of arithmetic operations etc. - Helper function now has 2 parameters: no need for cbind() in caller File: corr.rwl.seg.R -------------------- - Additional parameter in yr.range() - First definition of segavg.cor was not used. Now removed. - Some unnecessary name assignments and conversions to data.frame removed File: detrend.series.R ---------------------- - Fixed a bug caused by incorrect syntax in named argument (was <- in 1.4.1 and 1.4.2, correct form is =). ModNegExp detrending works again. File: glk.R ----------- - Uses the complete argument name MARGIN instead of MAR File: helpers.R --------------- - Additional parameter in yr.range(). Used by all callers. File: rcs.R ----------- - Uses warning() instead of cat() in one (unlikely?, impossible?) error situation - Better checking of pith offset names - yr.range() called in "the big for loop" instead of apply. The loop exists anyway, and there's no need to keep yr.range() of all series at the same time. - rwca is no longer a data.frame, and doesn't have unnecessary colnames File: read.tridas.R ------------------- - Fixed a bug where a wrong number of derived series was reported in summary output - A performance optimization for the case where ids.from.title is FALSE, ids.from.identifiers is TRUE (the defaults), and there are no identifiers File: series.rwl.plot.R ----------------------- - Uses complete argument names File: wavelet.plot.R -------------------- - Uses complete argument names - gettext() is used in some default values File: wavelet.plot.Rd --------------------- - Uses complete argument names - Default values match the changes in wavelet.plot.R File: write.crn.R ----------------- - Uses complete argument names * CHANGES IN dplR VERSION 1.4.2 * Ran through Mikko's changes. June 2, 2011. AGB * Changes below by Mikko Korpela All .rda data files (change reported June 7, after release in CRAN) ------------------- - Data files were repackaged by R-Forge using a tight compression level. This is done every time R-Forge builds the dplR package, but the files are expected to stay identical if no changes are made to the compression system. The new data files supposedly require R 2.10.0, but this does not affect dplR which requires R >= 2.11.0 anyway. File: ccf.series.rwl.R ---------------------- - as.vector() is more intuitive than c() with one argument - Cosmetic improvement File: ccf.series.rwl.Rd ----------------------- - bin.floor must be non-negative, not necessarily positive File: cms.R ----------- - Removed redundant c() - Some values are stored for reuse File: corr.rwl.seg.R -------------------- - par(op) moved to on.exit() - Odd and even segs are plotted with the same code, avoiding copy-paste - Some values are stored for reuse - Some unnecessary colnames are not set - Cosmetic improvement File: corr.rwl.seg.Rd --------------------- - bin.floor must be non-negative, not necessarily positive File: corr.series.seg.Rd ------------------------ - bin.floor must be non-negative, not necessarily positive File: crn.plot.R ---------------- - Default value of f is explicitly 0.5 File: crn.plot.Rd ----------------- - Clarified the default values of f and nyrs File: ffcsaps.R --------------- - Cosmetic improvement File: i.detrend.series.R ------------------------ - A value is stored for reuse - Cosmetic change File: qa.xdate.R ---------------- - bin.floor must be non-negative, not necessarily positive - Checks that bin.floor is non-negative. File: rcompact.c ---------------- - Accepts comment lines in the beginning of the file - Cosmetic changes (formatting of comments) - UNPROTECT all PROTECTed structures at the same time File: rcs.R ----------- - Some values are stored for reuse File: read.compact.R -------------------- - Prints comments found in the file (if any) File: rwi.stats.R ----------------- - Within-tree signal is now computed correctly (thanks to Pierre Mérian) File: rwl.stats.R ----------------- - Removed redundant instances of c() - Cosmetic improvement File: series.rwl.plot.Rd ------------------------ - bin.floor must be non-negative, not necessarily positive File: skel.plot.R ----------------- - Only makes as many viewports as needed - Some values are stored for reuse - Checks that length of input exceeds a minimum value - Avoids warning from giving an empty vector to range() File: wavelet.plot.R -------------------- - Some lines previously in both branches of if-else now appear only once - Avoids a duplicate call to unique() 2011-05-26 Andy Bunn * CHANGES IN dplR VERSION 1.4.1 * All changes by Mikko Korpela File: DESCRIPTION ----------------- - Minimum R version is now 2.11.0 (previously not specified). Reason behind the requirement: The encoding support added to read.tucson brought out a bug in read.fwf. The bug exists in 2.10.1 but is fixed in 2.11.0. - Suggests foreach and iterators (used in some functions if installed) Files: bai.in.R, bai.in.Rd, bai.out.R, bai.out.Rd ------------------------------------------------- - Cosmetic improvement - Also changed indentation of .R files to the recommended 4 spaces, as in the other .R, .c and .h files edited in this patch set. File: ccf.series.rwl.R ---------------------- - || instead of | - Optimized the order of expressions in a chain of the form x_1 || ... || x_n (order did not matter when | was used) - Removed an unused variable. - Cosmetic improvement File: ccf.series.rwl.Rd ----------------------- - Cosmetic improvement File: chron.R ------------- - Removed redundant c() - Default value of prefix is now "xxx". Previously the default value was NULL, which was then converted to "xxx". - Cosmetic improvement File: chron.Rd -------------- - Documents new (explicit) default value of prefix. - Cosmetic improvement File: cms.R ----------- - Some added robustness against weird input - Optimizations - Cosmetic improvement File: cms.Rd ------------ - Note about the requirement that the years be increasing and continuous File: combine.rwl.R ------------------- - Input handling was improved. - Loops were eliminated from combinator() File: combine.rwl.Rd -------------------- - Note about requirements for input (also existed in previous versions) - Author of the patch mentioned File: corr.rwl.seg.R -------------------- - || instead of | - Optimized the order of expressions in a chain of the form x_1 || ... || x_n (order did not matter when | was used) - Small optimizations - Cosmetic improvement File: corr.series.seg.R ----------------------- - Cosmetic improvement File: crn.plot.R ---------------- - Cosmetic improvement File: detrend.R --------------- - Under some circumstances, the function now uses the foreach / %dopar% structure from the foreach package. Conditions for this to happen: foreach must be installed, and at least one of the more time-consuming detrending methods must be used ("Mean" doesn't qualify). For a speed gain, the user must register a parallel backend for foreach. - Cosmetic improvement File: detrend.Rd ---------------- - Added notes about foreach and the detrending methods used by default. File: detrend.series.R ---------------------- - && instead of & - Now uses match.arg. - Cosmetic improvement File: detrend.series.Rd ----------------------- - Added a note about the detrending methods used by default. File: exactmean.c ----------------- - Removed some explicit type casts. - Doesn't use (pointless) dynamic memory allocation anymore. File: exactmean.R ----------------- - Converts the return value of length() to an integer: "programmers should not rely on it" (i.e. the return value may not be an integer). File: exactsum.c ---------------- - Makes use of dplr_double instead of long double. File: exactsum.h ---------------- - Defines and explains the new type dplr_double (for now, same as long double). File: ffcsaps.R --------------- - Tweaked error messages (should -> must) - Cosmetic improvement File: gini.c ------------ - Uses floating point constants instead of integer constants where appropriate. - Some pointless dynamic memory allocations were removed (replaced by statically allocated memory). File: gini.coef.R ----------------- - Converts the return value of length() to an integer. File: glk.R ----------- - Removed an unused variable. - Cosmetic improvement File: helpers.R --------------- - Removed unused matching functions, added a function that is actually used (moved from read.tridas.R) - Also moved some other functions here File: i.detrend.R ----------------- - Fixed a bug where arguments nyrs, f, and pos.slope were not being passed to i.detrend.series. - Cosmetic improvement File: i.detrend.series.R ------------------------ - Fixed a bug where arguments nyrs, f, and pos.slope were not being passed to detrend.series. - Cosmetic improvement File: morlet.R -------------- - Commented out the unused variables. - Cosmetic improvement File: normalize.xdate.R ----------------------- - Cosmetic improvement File: normalize1.R ------------------ - Cosmetic improvement File: qa.xdate.R ---------------- - Cosmetic improvement File: rcompact.c ---------------- - Some pointless dynamic memory allocations were removed (replaced by statically allocated memory). File: rcs.R ----------- - Cosmetic improvement File: read.compact.Rd --------------------- - New entries in "See Also" File: read.crn.R ---------------- - Streamlined file handling code. - Now has a parameter for the encoding used in the data file. This makes it possible to read files with any character encoding supported by R (may vary between R installations). - Removed an unused variable. - Improved "if" constructs (|| instead of |, evaluation only until first TRUE) - Cosmetic improvement File: read.crn.Rd ----------------- - Documents the new encoding parameter. File: read.fh.R --------------- - Fixed one if condition. - Now prints a summary of the data.frame, like the other read functions. - Streamlined code - Cosmetic improvement File: read.fh.rd ---------------- - Author of the patch mentioned File: read.ids.R ---------------- - Streamlined code - Cosmetic improvement File: read.ids.Rd ----------------- - Author of the patch mentioned File: read.rwl.R ---------------- - Automatic detection of file type now includes TRiDaS and Heidelberg formats. - Now uses match.arg and a switch expression to select the right branch. File: read.rwl.Rd ----------------- - Shows the new default value of the format parameter (related to the use of match.arg). - Lists TRiDaS and Heidelberg formats. - Removed some "See" links from "Details", because there is a separate "See Also" section, and the "See" links did not serve a specific purpose. - Describes the different return values when reading different types of data files. - Literal quotes were replaced with \dQuote. File: read.tridas.R ------------------- - Spelling change "meter" -> "metre" - Helper functions moved elsewhere to make the file shorter. - Changed the environment where some functions are defined. - All end tag handlers now call function end.element. Makes the code shorter but increases running time. Now xgettext does not complain about read.tridas being too long. It might be good to pay attention to the recommendations concerning diagnostic messages at http://cran.r-project.org/doc/manuals/R-exts.html#Diagnostic-messages File: read.tridas.Rd -------------------- - New entries in "See Also" - Spelling change "meter" -> "metre" File: read.tucson.R ------------------- - Now has a parameter for the encoding used in the data file. This makes it possible to read files with any character encoding supported by R (may vary between R installations). - New fix.internal.na copes with successive NAs and vectors with less than 2 elements - Cosmetic improvement File: read.tucson.Rd -------------------- - New entries in "See Also" - Documents the new encoding parameter. File: readloop.c ---------------- - Uses floating point constants instead of integer constants where appropriate. File: rwi.stats.R ----------------- - && instead of & - Now uses match.arg for 'period'. - An additional check is applied to the input. - Cosmetic improvement File: rwi.stats.Rd ------------------ - Shows the (technically) new default value of 'period'. File: rwi.stats.running.R ------------------------- - An additional check is applied to the input. - If the foreach package is installed, the function now uses it and %dopar% for executing the loop that iterates over the various running windows. If a parallel backend for foreach has been registered, this should give a nice speedup. - Removed some unnecessary calls to inc, where the conventional from:to is guaranteed to succeed (to > from). - Fixed an off-by-one error in max.offset. - Now uses match.arg for 'period'. File: rwi.stats.running.Rd -------------------------- - Note about using the foreach package. - Shows the (technically) new default value of 'period'. File: sea.R ----------- - In warnings, we use multiple arguments instead of paste. - select.y() was removed (rows of data.frames can be indexed with character strings). - Number of nested for loops was reduced. - Unnecessary computations were removed. - Added a check that the input x is a data.frame. - Check whether 'se' is NA. - Streamlined code. File: sea.Rd ------------ - Author of the patch mentioned File: sens.c ------------ - Uses floating point constants instead of integer constants where appropriate. - Some pointless dynamic memory allocations were removed (replaced by statically allocated memory). Files: sens1.R, sens2.R ----------------------- - Converts the return value of length() to an integer. File: series.rwl.plot.R ----------------------- - Removed unused variables. - Cosmetic improvement File: series.rwl.plot.Rd ------------------------ - Author of the patch mentioned File: skel.plot.R ----------------- - && instead of & - Unnecessary third column of 'skel.df' removed - Cosmetic improvement File: skel.plot.Rd ------------------ - Cosmetic change File: spag.plot.R ----------------- - on.exit() is used for resetting graphical parameters - Cosmetic improvement File: tbrm.R ------------ - Converts the return value of length() to an integer. File: tbrm.c ------------ - No longer uses long doubles in places where they are not really needed. May slightly affect some computation results. - Removed some explicit type casts. - Removed two temporary variables. Now the expressions used for computing them are used directly in the proper place. - Uses floating point constants instead of integer constants where appropriate. - Uses right shift instead of division by two where appropriate. - A relatively pointless dynamic memory allocations was removed (replaced by statically allocated memory). File: wavelet.plot.R -------------------- - Removed and commented out unused variables - Default value of 'f' is now explicitly 0.5. - Some cosmetic / readability improvements File: wavelet.plot.Rd --------------------- - Default value of 'f' reported differently. - Cosmetic improvements File: wc.to.po.R ---------------- - The function was moved from read.tridas.R to a new file. File: write.compact.R --------------------- - || instead of | - Uses on.exit() for closing the file - Cosmetic improvements File: write.compact.Rd ---------------------- - New entry in "See Also" File: write.crn.R ----------------- - Removed unused variables. - Streamlined code - Cosmetic improvement File: write.crn.Rd ------------------ - Max Width of Site ID is 6, not 5 - Some typos fixed File: write.rwl.R ----------------- - Added the option to write in the TRiDaS format. - Now uses match.arg and a switch expression to select the right branch ('format'). File: write.rwl.Rd ------------------ - Added notes related to the newly added support for the TRiDaS format. - Removed some "See" links from "Details", because there is a separate "See Also" section, and the "See" links did not serve a specific purpose. - Shows the new default value of the format parameter. - Literal quotes were replaced with \dQuote. - New entry in "See Also" File: write.tridas.Rd --------------------- - Spelling change "meter" -> "metre" - New entries in "See Also" - Cosmetic improvement File: write.tucson.R -------------------- - Reduced the number of names in the dplR name space by wrapping the creation of the format.tucson list inside an anonymous function. - Removed unused variables. - Streamlined code - Cosmetic improvement - Uses on.exit() for closing the file - Uses || instead of | File: write.tucson.Rd --------------------- - New entry in "See Also" 2011-05-18 Andy Bunn * CHANGES IN dplR VERSION 1.4.0 * Incorporation of TRiDAS via Mikko Korpela, May 16, 2011 File: NAMESPACE --------------- New imports (digest, xmlEventParse from XML) and exports (read.tridas, write.tridas, uuid.gen, po.to.wc, wc.to.po, tridas.vocabulary). File: po.to.wc.Rd ----------------- New documentation file. File: read.tridas.R ------------------- New file. Function for reading TRiDaS format files. File: read.tridas.Rd -------------------- New documentation file. File: simpleXML.R ----------------- Fast function for writing XML files. Used in write.tridas, not exported to user. File: tridas.vocabulary.R ------------------------- New file. Function for browsing TRiDaS vocabulary. File: tridas.vocabulary.Rd -------------------------- New documentation file. File: uuid.gen.R ---------------- The new function uuid.gen creates a generator of Universally Unique IDentifiers (UUIDs). It uses the digest package, making it a new dependency of dplR. The code in uuid.gen itself is very simple. This is a very generally applicable function. It is used in write.tridas, but may also be useful to R users in general. The synchronicity package in CRAN provides similar functionality (and more), but is not available for Windows. Also, the UUID generator returned by uuid.gen seems to be faster than the uuid function in synchronicity. More specifically, the latter uses much more system time. The difference in user time is small (in favor of uuid of synchronicity). File: uuid.gen.Rd ----------------- New documentation file. File: wc.to.po.Rd ----------------- New documentation file. File: write.tridas.R -------------------- New file. Function for writing TRiDaS format files. File: write.tridas.Rd --------------------- New documentation file. 2010-11-15 Andy Bunn * CHANGES IN dplR VERSION 1.3.9 * Added an option to control the colors in plot.wavelet(). * Added an option to plot wavelets side by side instead of stacked (plus a few other beautifications). * Changes mostly in wavelet.plot.R and arg side.by.side and key.col added to the help. * Added an axis on side three for the wavelet plot. 2010-10-27 Andy Bunn * CHANGES IN dplR VERSION 1.3.8 all from Mikko Korpela File: corr.rwl.seg.R -------------------- * Changed the code that computes the first year of the first segment. Now it doesn't waste data in the case that the first year of the data is at a multiple of bin.floor. The same change was made to corr.series.seg.R in an earlier patch (dplR 1.3.3). Previously: else min.bin = min.yr%/%bin.floor*bin.floor+bin.floor Now: else min.bin = ceiling(min.yr/bin.floor)*bin.floor * The size of the bins is now seg.length, not seg.length+1. * Fixed some (potential) errors caused by R dropping dim() when subsetting an array. The fix was to add drop=FALSE. * Small optimizations in the style of an earlier patch to corr.series.seg.R. * Replaced & with && in some if conditions. * Adjusted the indentation of the source code. * Added a check requiring >= 2 series in rwl. File: corr.rwl.seg.Rd --------------------- * Updated the min.yr, bin.floor code excerpt included in the document. File: corr.series.seg.R ----------------------- * The size of the bins is now seg.length, not seg.length+1. * Similarly, what was previously to = max(series.yrs)-seg.length-seg.length is now to = max(series.yrs)-seg.length-seg.length+1 File: crn.plot.R ---------------- * Moved resetting of graphical parameters to on.exit. As a result, resetting also works in case of an unexpected error that terminates the function. * Reworked the structure of the function so that lines of code that are common to different branches (sample depth or no sample depth) are not replicated. Also removed special treatment of the case of nCrn==1. These changes have no effect on the output (the plot). * Other tweaks also, mostly cosmetic. File: detrend.series.Rd ----------------------- * Correction to documentation: If only one method is selected, the function returns a vector. File: ffcsaps.R --------------- * An ugly work-around (test and transpose) for unwanted dimension dropping was replaced by drop=FALSE. Why didn't I discover this solution earlier... File: helpers.R --------------- * Added a few new helper functions: fix.names is (or will be) used in a few functions, and some other functions are "helpers of the helper". Compared to the old fix.names (previously in write.tucson.R), the new one has the option of unlimited name length, and the option of limiting the set of allowed characters to the union of LETTERS, letters, and 0:9, where letters (LETTERS) are the lowercase (uppercase) letters of the English alphabet. * The new is.int tests if numbers have integer value. File: morlet.R -------------- * Added drop=FALSE to a few matrix subset operation, just to be on the safe side (later it's assumed that wave is a matrix). I don't know if the danger of a dimension drop (conversion to vector) could have materialized with any (valid) inputs. File: normalize.xdate.R ----------------------- * drop=FALSE File: qa_xdate.R ---------------- * There is now a check that catches negative seg.length. * The helper function is.int previously found in this file is now in helpers.R (completely rewritten). File: rcs.R ----------- * Removed the internal is.int function. The main function now uses the new version of is.int, located in helpers.R, which works not only for single numbers, but for longer vectors and arrays also. This upgrade in functionality is necessitated by the way the function is used in rcs.R. * Fixed a bug, where the function only checked that the first pith offset is integer. Now checks all pith offsets. File: read.crn.R ---------------- * Like rcs.R, also this function now uses the is.int located in helpers.R. * drop=FALSE was applied to many expressions. File: read.tucson.R ------------------- * drop=FALSE * no more as.matrix in many expressions File: rwi.stats.running.R ------------------------- * rwi is transformed to matrix form at an early stage, which may result in faster operation. * drop=FALSE * no more as.matrix in many expressions File: seg.plot.R ---------------- * Does not crash in the case of ncol(rwl) == 1. * Resets graphical parameters on exit, even in error cases. File: spag.plot.R ----------------- * Fixed plotting of a one-series data.frame. * Previously, the number of columns in rwl was computed with both dim(rwl)[2] and ncol(rwl). These are equivalent. Only one of them remains. * Checks that at least one series was given, otherwise stops. * Computes y axis limits more sensibly: due to overlap being allowed in the plot, the maximum value may occur in a series other than the one that is ordered to be drawn on the top of the spaghetti plot. Also, the previous upper limit setting computed the maximum over a set of one value (which is wrong) that was possibly NA (which would crash the function). A plausible explanation for the origin of this bug is the conversion from data.frame to matrix performed by the scale function. This was probably not taken into account when indexing rwl with rwl[nseries]. File: write.compact.R --------------------- * Now produces files with DOS / Windows / etc. line terminators, CR+LF, on all platforms. If this has any effects on compatibility with old software (not sure about that), I expect the positives to outweigh the negatives. Also, files with Unix (LF) line terminators (previously created by dplR on Linux and other systems using LF) may show as one long line with primitive text editors in Windows. * The function now enforces a maximum length for series names differently: does not stop and print an error, but truncates the name. Also, the function no longer stops when characters outside the ranges a-z, A-Z, 0-9 are found, but fix.names drops the offending characters. The previous regular expression used for testing series names may have accepted non-ASCII letters, but fix.names does not (with basic.charset == TRUE). The mapping from original names to names created by fix.names can be written to a file (two new parameters to the function). * Replaced one as.data.frame with a solution based on drop=FALSE. File: write.compact.Rd ---------------------- * Added descriptions of the new parameters of the function. The descriptions are the same as in write.tucson.Rd. * Added information about allowed series ids and automatic renaming. * Author info now correctly refers to write.tucson instead of write.rwl. File: write.tucson.R -------------------- * The function no longer stops when characters outside the ranges a-z, A-Z, 0-9 are found, but fix.names drops the offending characters. * Any automatic changes to series ids result in a warning or several warnings. Also, if the option to write id changes to a file is used, all the changes are written to the file. Previously, the excess tail of ids could be silently cut off. * Now produces files with DOS / Windows / etc. line terminators, CR+LF, on all platforms. See write.compact.R. * Removed the unused variable hdr. * Moved some helper functions to helpers.R. * Replaced one as.data.frame with a drop=FALSE solution. File: write.tucson.Rd --------------------- * Adjusted the description of parameter mapping.fname: Now all changed series ids are printed, also the ones that were only truncated. * Adjusted the part of the Details section that describes automatic editing of series ids. 2010-09-21 Andy Bunn * CHANGES IN dplR VERSION 1.3.7 * Modified wavelet.plot() to produce a better looking plot and put the wavelet code into a new function called morlet(). * Deprecated cwt.filled.contour * Added normalize1 to NAMESPACE * Added argument label.cex to pass to axis to increase the size of the labels on the plot in corr.rwl.seg() and modified the help file accordingly. This change was requested by a user and seems like a good idea. * Corrected some spelling in help for corr.rwl.seg(), ccf.series.seg(), and corr.series.seg(). I'm sure there is more. 2010-07-17 Andy Bunn * CHANGES IN dplR VERSION 1.3.6 * Modified rcs() to stop() if any of the pith offsets are <1 or not integers. The help file indicated that this should be so but it's now explicit in the code. * Fixed a bug in corr.rwl.seg() where floating series were not being drawn properly - the plot had them displaying all the way to the max yr range b/c of an indexing problem. Also made some cosmetic changes to the guides in between segments for the graph. I do NOT like the way the plot on this function is done with the odd bin and even bin looping. Very clunky and confusing to look at. The correlation part is fine and the graph is nice but the logic of the plot is silly. * Clarified language the help page for corr.rwl.seg(). * Fixed a bug in corr.series.seg() where short series coupled with long segment lengths caused the function to crash. I added to the fix Mikko made in 1.3.3 by requiring that the minimum series length must be twice the segment length. Otherwise the function is useless. Error message now indicates this as does the help file. Made some graphical improvements with bin lines extending past the actual bins to better match the moving correlation line. * Clarified language the help page for corr.series.seg(). * Fixed a bug in ccf.series.rwl() where panels were out of order on series with long time spans. This involved reordering the ccf.df$bin factor to be the same order as bins[,1]. * Clarified language the help page for ccf.series.rwl(). * Improved series.rwl.plot() to give more diagnostic information in four plots. * Improved the help page for series.rwl.plot() to better explain what it now does. 2010-07-05 Andy Bunn * CHANGES IN dplR VERSION 1.3.5 * Modified the help file for write.crn() so that the elev in the hdr will be written in a way that might be more robust in terms of the ITRDB standard. But, it's very clunky and silly to depend on an unreliable standard. I also added some text to the help file for write.tucson() to indicate that the header should be thought of as experimental. Thank goodness Mikko is working on getting tridas implemented. * Fixed a bug in rcs() where the rc was calculated wrong if none. The function implicitly assumed that at least one of the pith offsets would be 1. When all the pith offsets were >1 the function would crash b/c of bad indexing. 2010-05-13 Andy Bunn * CHANGES IN dplR VERSION 1.3.4 * fixed a bug in detrend() where args nyrs and f were not being passed to detrend.series(). * Corrected at typo in Zang's name. (So sorry Christian!) 2010-04-22 Andy Bunn * CHANGES IN dplR VERSION 1.3.3 made by Mikko Korpela * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: NAMESPACE --------------- Added rwi.stats.running to exports. * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: bai.in.R -------------- Removed one for-loop by transforming it to vector operations. Also some other optimizations (move some operations outside the remaining for-loop, streamline the writing of the results). NOTE: Why is (was) there a NaN check in calc.area? I removed it as pointless. Also note that NaN is different from NA (in my opinion, NA check would also be pointless). As calc.area became quite simple after my edits, I moved the remaining code to the main function. * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: bai.in.Rd --------------- Added my name to it. * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: bai.out.R --------------- Similar optimizations as to bai.in.R. One particular optimization was to replace max(cumsum(dat2)) with sum(dat2), which gives the same result (but faster) when there are no negative values, as in ring width series. * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: bai.out.Rd ---------------- Added my name to it. * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: chron.R ------------- Now uses rowMeans (rowSums), which is faster than computing the means (sums) with apply. Now uses ar.func defined in helpers.R. * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: chron.Rd -------------- Added my name to it. * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: cms.R ----------- Changed the sequence notation i:i to just i. Removed the pointless variable err3. Removed the err1 and err2 arrays, using scalar variables instead (only need one value at a time). The computation of err2 is now more efficient, and err2 is of opposite sign than before (taken into account when the value is utilized). Compute err6 with vector operations instead of using a for loop. Use fewer multiplications in the computation of err6 by using the common factor, sqrt(med). Removed pointless call to the function c in c.curve <- c(index[,2]). Avoid searching for NA values multiple times, save the results of the search in the variable no.na. Edited the if/else structure in the end. Now it makes fewer tests than before. Removed yr.range and sortByIndex, now uses those defined in helpers.R. * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: cms.Rd ------------ Added my name to it. * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: corr.series.seg.R ----------------------- Now produces a more useful error message if the data set is so small that no segments fit over the data. Example based on the example in the .Rd file: > data(co021) > dat=co021 > #create a missing ring by deleting a year of growth in a random series > flagged=dat$'641143' > flagged=c(NA,flagged[-325]) > names(flagged)=rownames(dat) > dat$'641143'=NULL > dat=dat[(nrow(dat)-48):nrow(dat),] # clip > flagged=flagged[(length(flagged)-48):length(flagged)] # clip > seg.24=corr.series.seg(rwl=dat,series=flagged,seg.length=24,biweight=FALSE) produces this message in dplR 1.3.1 (.2 also, I think): Error in seq.default(from = min.bin, to = max(series.yrs) - seg.length, : wrong sign in 'by' argument but after a change to the code of the function: Error in corr.series.seg(rwl = dat, series = flagged, seg.length = 24, : Cannot fit any segments (not enough years in the series) I also made a mostly cosmetic change (got rid of bins1 and bins2), and made the following functional change: Previously: else min.bin = min(series.yrs)%/%bin.floor*bin.floor+bin.floor Now: else min.bin = ceiling(min(series.yrs)/bin.floor)*bin.floor The example below shows how the new code line produces a less wasteful result (min.bin1 below) than the previous one (min.bin2), which jumps to the next level "one year too early", and consequently misses a segment that would have been possible to include. > min.year<-1900:2000 > min.bin1 <- ceiling(min.year/25)*25 > min.bin1 [1] 1900 1925 1925 1925 1925 1925 1925 1925 1925 1925 1925 1925 1925 1925 1925 [16] 1925 1925 1925 1925 1925 1925 1925 1925 1925 1925 1925 1950 1950 1950 1950 [31] 1950 1950 1950 1950 1950 1950 1950 1950 1950 1950 1950 1950 1950 1950 1950 [46] 1950 1950 1950 1950 1950 1950 1975 1975 1975 1975 1975 1975 1975 1975 1975 [61] 1975 1975 1975 1975 1975 1975 1975 1975 1975 1975 1975 1975 1975 1975 1975 [76] 1975 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 [91] 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 > min.bin2 <- min.year%/%25*25+25 > min.bin2 [1] 1925 1925 1925 1925 1925 1925 1925 1925 1925 1925 1925 1925 1925 1925 1925 [16] 1925 1925 1925 1925 1925 1925 1925 1925 1925 1925 1950 1950 1950 1950 1950 [31] 1950 1950 1950 1950 1950 1950 1950 1950 1950 1950 1950 1950 1950 1950 1950 [46] 1950 1950 1950 1950 1950 1975 1975 1975 1975 1975 1975 1975 1975 1975 1975 [61] 1975 1975 1975 1975 1975 1975 1975 1975 1975 1975 1975 1975 1975 1975 1975 [76] 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 [91] 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2025 * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: corr.series.seg.Rd ------------------------ Added my name to it. * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: helpers.R (new) --------------- A couple of helper functions to work around a design stupidity of R: http://radfordneal.wordpress.com/2008/08/06/design-flaws-in-r-1-reversing-sequences/ Also moved ar.func here. The function was previously defined in three different places, and always functionally the same. I also optimized the function a tiny bit. So, if any changes to the function are made in the future, they will show up in every "client" function without having to edit three+ different files. Moved yr.range here. It is a slightly optimized version of the function used in cms, rcs, rwl.stats, seg.plot, and spag.plot. There's still a local definition for yr.range in corr.rwl.seg, because the function there is a bit different. Moved sortByIndex here. It is a slightly optimized version of the function used in cms and rcs. Is this a good place for the helper functions? If you have a better suggestion, feel free to move the functions or rename the file. * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: ffcsaps.R --------------- Fixed the helper function spdiags. The previous version crashed ffcsaps in the marginal case where it was given a data set with just three points. The bug was kind of related to the design stupidity just mentioned above. Now the helper function is documented, and uses the function "inc" defined in helpers.R. The use of the inc function avoids the problematic case where the sequence max(1,1-d.k):min(n,n-d.k), which denotes row numbers in the sparse matrix, might include zero. The "inc" function ensures that we get an increasing (or empty) sequence (does not include zeros when we start above zero), as probably was originally intended. I also made the code a bit clearer by renaming variables. I wonder what the kludgy part at the end of ffcsaps actually does? Whose code is that? Could it be rewritten in a non-kludgy way and/or with some more comments? What purpose does the variable x2 serve, and why does it contain a fixed-interval sequence with a seemingly arbitrary length of 101? The kludgy part sometimes prints a warning. * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: glk.R ----------- Small patch. Avoids a crash in the unlikely scenario that the function is called with an x consisting of just one record. After the fix, the function returns (expectedly, but not too usefully) NA. Also removed the unused internal function strip.na(x). * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: glk.Rd ------------ Added my name to it. * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: normalize.xdate.R ----------------------- Changes comparable to normalize1.R. * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: normalize1.R ------------------ Now uses colMeans instead of apply in one statement, and colSums instead of apply in another statement. Uses ar.func defined in helpers.R. * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: plot.series.rwl.R ----------------------- Save temporary results for reuse. One line / statement removed (the result was not used). * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: rcs.R ----------- Use rowMeans. Removed yr.range and sortByIndex, now uses those defined in helpers.R. * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: rcs.Rd ------------ Added my name to it. * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: read.tucson.R ------------------- Fixed a bug concerning the width of the decade field in the case long=FALSE. The correct choice is 4, and 8+4 makes 12, which is consistent with 7+5=12 in the case long=TRUE, and the Tucson format description (column 13 is already ring width data). The bug would have caused trouble in the presence of 6 digit ring widths. * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: rwi.stats.R ----------------- Fixed a typo (seies -> series), another typo (Correaltions -> Correlations). Reorganized the code a bit. Maybe this should be fixed to use number of trees instead of number of cores in the EPS calculation. Also, I think it's dangerous to divide a sum of correlations by the theoretical number of terms in the sum, because sum of the terms may end up being NA. Then the average of non-NA elements, computed by dividing with the theoretical number of terms, will be too small. * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: rwi.stats.Rd ------------------ Added my name to it. Changed the "see also" (unintentional?) self-reference into a reference to rwi.stats.running. Should the reference have been to rwl.stats? * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: rwi.stats.running.R ------------------------- Compute stats in a running window. New function. Computes the EPS a bit differently compared to rwi.stats (see comments to rwi.stats.R above). Quite slow. Optimizations or (partial?) rewrite in C would be welcome. Maybe some day. The terminology used in the function arguments and the documentation may need to be made consistent with other parts of dplR. I'm not the best expert on that topic. * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: rwi.stats.running.Rd -------------------------- Documentation for the new function. Based on a copy-paste from rwi.stats.Rd. * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: rwl.stats.R ----------------- Removed yr.range, now uses the version defined in helpers.R. * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: seg.plot.R ---------------- Removed yr.range, now uses the version defined in helpers.R. * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: skel.plot.R ----------------- Save temporary results for reuse. * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: skel.plot.Rd ------------------ Added my name to it. * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: spag.plot.R ----------------- Save temporary results for reuse. Removed yr.range, now uses the version defined in helpers.R. * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: spag.plot.Rd ------------------ Added my name to it. * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: wavelet.plot.R -------------------- Use multiplication instead of exponentiation in the computation of the power spectrum. This reduces computation time. Fixed the messy indentation of the source code. * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: wavelet.plot.Rd --------------------- Added my name to it. * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: write.crn.R ----------------- Fixed two bugs: one affected lines with only one year, the other affected the last line in case it was already complete (10 years). Example of the effect of the first bug (last line in file): xxxstd1990 540 12NANA 540 129990 09990 09990 09990 09990 09990 09990 09990 09990 0 After fix: xxxstd1990 540 129990 09990 09990 09990 09990 09990 09990 09990 09990 0 Example of the effect of the second bug (last line in file): xxxstd1990 540 12 554 12 499 12 262 12 502 12 435 12 472 12 513 12 538 11 679 119990 09990 0 After fix: xxxstd1990 540 12 554 12 499 12 262 12 502 12 435 12 472 12 513 12 538 11 679 11 The fixes involved changing for(j in 2:n.yrs) to for(j in inc(2,n.yrs)) and for(k in 1:yrs.left) to for(k in inc(1,yrs.left)) where inc(from,to) is a function in helpers.R. * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: write.tucson.R -------------------- Added a new function parameter. Setting long.names=TRUE allows series IDs with 7 or 8 characters depending on whether there are long year numbers in the data. * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ File: write.tucson.Rd --------------------- Describe the new parameter. (Also) files not named here --------------------------- Added argument names to every call of the seq function. Quote from ?seq: The interpretation of the unnamed arguments of 'seq' and 'seq.int' is _not_ standard, and it is recommended always to name the arguments when programming. I didn't take credit just for these small changes (no changes to .Rd files). 2010-04-12 Andy Bunn * CHANGES IN dplR VERSION 1.3.2 * typos in Rd files fixed * bugs fixed in write.tucson() and write.compact() where upper case IDs would cause the functions fail 2010-03-26 Andy Bunn * CHANGES IN dplR VERSION 1.3.1 * ChangeLog introduced. I am going to compile a full changelog from the src files of all the previous dplR versions from 1.0 forward. I've been remiss in not keeping one! * glk - New function. Gleichlaeufigkeit by Zang is a classical agreement test * sea - New function. Superposed epoch analysis by Zang * combine.rwl - updated by Zang. Now takes lists as input.