frictionless now supports Data Packages using the v2 specification, while maintaining support for those using v1. When creating a package, resource or schema, it uses the v2 specification.
create_package(), add_resource() and
create_schema() create a v2 package, resource and schema
respectively. This can be a breaking change for some
workflows! Also note that this can lead to mixed versions
(e.g. a v1 package with a v2 resource).upgrade_package() can be used to upgrade a package, its
resources and verbose schemas from v1 to v2 (harmonizing mixed versions)
(#376, #357, #343, #363).read_resource() and schema() no longer
require "profile": "tabular-data-resource" (nor
"type": "table"), which was the main blocker for reading
resources defined in the v2 specification. They still expects a
schema (#343).read_resource() forbids reading files from a path
containing a hidden
directory (#359).read_resource() supports labelled
missing values (#354).read_resource() supports integer groupChar
(#364).read_resource() supports the new list
field type. It interprets this as character (#368).read_resource() supports datetimes with optional
milliseconds when "format": "default" (#347).create_package() now creates a v2
package by setting $schema
to the recommended v2 value
("https://datapackage.org/profiles/2.0/datapackage.json")
(#349).create_schema() now creates a v2
schema by setting $schema
to the recommended v2 value
("https://datapackage.org/profiles/2.0/tableschema.json")
(#351).add_resource() now creates a v2
resource by setting $schema
to the recommended v2 value
("https://datapackage.org/profiles/2.0/dataresource.json")
and type
to "table". It no longer sets profile. If not
provided by the user, it will also create a v2 schema. Since it
typically does not define a dialect, it will default to a v1
dialect (#343, #356).add_resource() now allows any
string as resource_name. It trims leading and trailing
spaces from the name (#344).read_package() can read datapackage.json
defined in the v1 and v2 specification (#336).example_package() now uses the v2 specification as
default (#360, #361).{frictionless}’ support for v2
properties. Notably, schema.fieldsMatch
and field.categories
are not yet supported (#367).add_properties() allows to add properties to a Data
Package, Data Resource, Table Dialect or Table Schema (#382).version() is now generic and can report what version of
the Data Package standard is used by a Data Package (as before), Data
Resource, Table Dialect and Table Schema (#341).version() determines what version of the Data
Package standard is used by a Data Package (e.g. "1.0",
"2.0", ">=2.0"), based on the presence and
value of the $schema property (#299). This information is
also returned by print() (#302).read_package() now warns when reading a
datapackage.json that uses a version of the Data Package
standard not supported by frictionless (i.e. anything other than version
"1.0") (#309).read_resource() no longer guesses the type for fields
without a type, but sets it to character (the default for a
CSV). This aligns with a clarification in the specification
(#296).read_resource() now supports reading from remote zip
files, thanks to support in {vroom} (1.3.0) (#291).add_resource() with replace = TRUE adds
the resource if there is none to replace, rather than throwing an error
(#273).add_resource() now retains the URL to a provided
schema, rather than including it verbosely (#305).write_package()’s overwrite behavior is now as intended
and documented in the function (#304, #313).write_package() now prints multiple
resource$path, resource$schema$missingValues
and field$constraints$enum on multiple lines in the
datapackage.json (#297).resources() is soft-deprecated, please use
resource_names() instead (#282).get_schema() is soft-deprecated, please use
schema() instead (#282).frictionless now relies on R >= 4.1.0 (because of an indirect
{vroom} dependency) (#291) and uses base pipes
(|> rather than %>%) (#292).
resource() is now a public function, making it
possible to get a resource object (i.e. a list) by its name. This is
especially useful if you want to implement reading functionality not
supported by frictionless (#303).
New resource()<- allows to overwrite a resource
in place (#314).
Internal frictionless properties package$directory
and resource$read_from are now attributes
attr(package, "directory") and
attr(resource, "data_location"). This separates them better
from public Data Package and Resource properties (#289). Saved
Data Package objects created with previous versions of frictionless will
show a deprecation warning (#293) and can be updated with
create_package(). If you use these internal properties in
your R package, then change them:
# Before
package$directory
resource <- frictionless:::get_resource(package, "resource_name") # Internal function
resource$read_from
# After
attr(package, "directory")
resource <- frictionless:::resource(package, "resource_name") # Function renamed
attr(resource, "data_location") # Attribute renamed"key": ["value"]) in a
datapackage.json file are now retained when reading and
writing (#276).read_package() now returns a warning rather than an
error when a datapackage.json contains no resources. This
allows use to create the JSON and then add resources with frictionless
(#265).example_package() now has a version
parameter, allowing to load the example Data Package following the Data
Package v1 or v2 specification (#249).add_resource() now allows to replace an existing
resource (#227).read_resource() now returns an error if both
path and data are provided (#143).write_package() no longer writes to "." by
default, since this is not allowed by CRAN policies. The user needs to
explicitly define a directory (#205).null values in a read datapackage.json are
now retained by write_package(), rather than being changed
to empty lists. Properties assigned by the user to NA and
NULL remain being written as null and removed
respectively (#203).vignette("data-package"),
vignette("data-resource"),
vignette("table-dialect") and
vignette("table-schema") describe how frictionless
implements the Data Package standard. The (verbose) function
documentation of read_resource() and
create_schema() has been moved to these vignettes,
improving readability and maintenance (#208, #246).example_package is removed in
favour of example_package(). This function allows to
reproducibly provide a local Data Package, without the need for an
internet connection. The observations resource was also
changed from a remote to a local resource and from CSV to TSV.
This change affects the use of example_package in
older versions of frictionless. We recommend to update
frictionless to the latest version (#114, #253).read_resource() is now more modular under the hood,
which should make it easier to extend (#210).CITATION.cff for citation and
Zenodo deposit (#206).print() prints a human-readable summary of the Data
Package, rather than a (long) list (#155).read_package() no longer returns a message regarding
rights and credit (#121). If package$id is a URL (e.g. a
DOI) it will be mentioned in print().add_resource() accepts additional arguments via
.... These are added as (custom) properties to the resource
and are retained in write_package() (#195).read_resource() now supports column selection via the
col_select argument from readr::read_delim().
This can vastly improve reading speed (#123). Tidy
selection is not supported.write_package() no longer adds
"profile": "tabular-data-package" to
datapackage.json. It is also removed from the example
dataset (#188).readr::problems() is included in NAMESPACE so you don’t
have to load readr to inspect parsing issues. The function is mentioned
in the documentation of read_resource() (#129).package) now has a
datapackage class (#184). This enables a custom
print() function (see above). check_package()
will warn if the class is missing, so previously saved Data
Package objects (without the class) will generate a
warning.check_package() is now a public function, so it can be
used in your package (#185). This and the other check_
functions return the first argument silently (rather than
TRUE), so they can be chained.create_package() now accepts a descriptor
argument so that a Data Package object can be created from an existing
object (#184). It will always validate the created object with
check_package().cli::cli_abort(), cli::cli_warn() and
cli::cli_inform() are used for all errors, warnings, and
messages (#163). This has several advantages:
{rlang} class,
e.g. frictionless_error_fields_without_name, making it
easier to test for specific errors.{glue} and {assertthat} are removed as
dependencies (#163). The functionality of glue is replaced by cli, while
assertthat::assert() calls are now if ()
statements.{rlang} is added as dependency (#192). It is already
used by other dependencies..zenodo.json can be created with
checklist::update_citation().{stringi} to Suggests. It was removed
as a dependency from {rmarkdown} 2.26, resulting in
“stringi package required for encoding operations” build errors on CRAN
(#176).skip_if_offline() to selected tests and verbosely
include output in vignette examples, to avoid CRAN errors caused by
timeouts (#116).datapackage.json, resource files, schemas) can
now be read from (s)ftp:// URLs (#102).add_resource() now sets format,
mediatype and encoding for added CSV file(s)
(#78).add_resource() now supports adding schema
via path or URL.write_package() now supports added data to be gzip
compressed before being written to disk (#98).read_resource() will now warn rather than error on
unknown encoding (#86).package objects no longer have or require the custom
attribute resource_names, use the new
resources() instead (#97).package objects no longer have or require the custom
attribute datapackage, making it easier to edit them as
lists (with e.g. append()).add_resource() now supports adding CSV file(s) directly
as a resource. This skips reading/handling by R and gives users control
over path (#74).example_package) are now downloaded when writing with
write_package(), rather than being skipped. This is more
consistent with locally read packages. The behaviour for resources with
a path containing URLs (only) and resources with
data remains the same (no files are written). The write
behaviour is better explained in the documentation (#77).write_package() now silently returns the output rather
than input package.create_package() will set
"profile": "tabular-data-package" since packages created by
frictionless meet those requirements (#81).create_schema() interprets empty columns as
string not boolean (#79).read_package() can now read from a
datapackage.yaml file.read_resource() now accepts YAML Table Schemas and CSV
dialects.add_resource()/create_schema()’s
df argument is renamed to data.example_package’s observations resource
now has URLs as path to serve as an example for that.