InferenceData

Type definition

InferenceObjects.InferenceDataType
InferenceData{group_names,group_types}

Container for inference data storage using DimensionalData.

This object implements the InferenceData schema.

Internally, groups are stored in a NamedTuple, which can be accessed using parent(::InferenceData).

Constructors

InferenceData(groups::NamedTuple)
InferenceData(; groups...)

Construct an inference data from either a NamedTuple or keyword arguments of groups.

Groups must be Dataset objects.

Instead of directly creating an InferenceData, use the exported from_xyz functions or convert_to_inference_data.

Property interface

Base.getpropertyFunction
getproperty(data::InferenceData, name::Symbol) -> Dataset

Get group with the specified name.

Base.propertynamesFunction
propertynames(data::InferenceData) -> Tuple{Symbol}

Get names of groups

Indexing interface

Base.getindexFunction
Base.getindex(data::InferenceData, groups::Symbol; coords...) -> Dataset
Base.getindex(data::InferenceData, groups; coords...) -> InferenceData

Return a new InferenceData containing the specified groups sliced to the specified coords.

coords specifies a dimension name mapping to an index, a DimensionalData.Selector, or an IntervalSets.AbstractInterval.

If one or more groups lack the specified dimension, a warning is raised but can be ignored. All groups that contain the dimension must also contain the specified indices, or an exception will be raised.

Examples

Select data from all groups for just the specified id values.

julia> using InferenceObjects, DimensionalData

julia> idata = from_namedtuple(
           (θ=randn(4, 100, 4), τ=randn(4, 100));
           prior=(θ=randn(4, 100, 4), τ=randn(4, 100)),
           observed_data=(y=randn(4),),
           dims=(θ=[:id], y=[:id]),
           coords=(id=["a", "b", "c", "d"],),
       )
InferenceData with groups:
  > posterior
  > prior
  > observed_data

julia> idata.posterior
Dataset with dimensions:
  Dim{:chain} Sampled 1:4 ForwardOrdered Regular Points,
  Dim{:draw} Sampled 1:100 ForwardOrdered Regular Points,
  Dim{:id} Categorical String[a, b, c, d] ForwardOrdered
and 2 layers:
  :θ Float64 dims: Dim{:chain}, Dim{:draw}, Dim{:id} (4×100×4)
  :τ Float64 dims: Dim{:chain}, Dim{:draw} (4×100)

with metadata OrderedCollections.OrderedDict{Symbol, Any} with 1 entry:
  :created_at => "2022-08-11T11:15:21.4"

julia> idata_sel = idata[id=At(["a", "b"])]
InferenceData with groups:
  > posterior
  > prior
  > observed_data

julia> idata_sel.posterior
Dataset with dimensions:
  Dim{:chain} Sampled 1:4 ForwardOrdered Regular Points,
  Dim{:draw} Sampled 1:100 ForwardOrdered Regular Points,
  Dim{:id} Categorical String[a, b] ForwardOrdered
and 2 layers:
  :θ Float64 dims: Dim{:chain}, Dim{:draw}, Dim{:id} (4×100×2)
  :τ Float64 dims: Dim{:chain}, Dim{:draw} (4×100)

with metadata OrderedCollections.OrderedDict{Symbol, Any} with 1 entry:
  :created_at => "2022-08-11T11:15:21.4"

Select data from just the posterior, returning a Dataset if the indices index more than one element from any of the variables:

julia> idata[:observed_data, id=At(["a"])]
Dataset with dimensions:
  Dim{:id} Categorical String[a] ForwardOrdered
and 1 layer:
  :y Float64 dims: Dim{:id} (1)

with metadata OrderedCollections.OrderedDict{Symbol, Any} with 1 entry:
  :created_at => "2022-08-11T11:19:25.982"

Note that if a single index is provided, the behavior is still to slice so that the dimension is preserved.

Base.setindexFunction
Base.setindex(data::InferenceData, group::Dataset, name::Symbol) -> InferenceData

Create a new InferenceData containing the group with the specified name.

If a group with name is already in data, it is replaced.

Iteration interface

InferenceData also implements the same iteration interface as its underlying NamedTuple. That is, iterating over an InferenceData iterates over its groups.

General conversion

InferenceObjects.convert_to_inference_dataFunction
convert_to_inference_data(obj; group, kwargs...) -> InferenceData

Convert a supported object to an InferenceData object.

If obj converts to a single dataset, group specifies which dataset in the resulting InferenceData that is.

See convert_to_dataset

Arguments

  • obj can be many objects. Basic supported types are:
    • InferenceData: return unchanged
    • Dataset/DimensionalData.AbstractDimStack: add to InferenceData as the only group
    • NamedTuple/AbstractDict: create a Dataset as the only group
    • AbstractArray{<:Real}: create a Dataset as the only group, given an arbitrary name, if the name is not set

More specific types may be documented separately.

Keywords

  • group::Symbol = :posterior: If obj converts to a single dataset, assign the resulting dataset to this group.

  • dims: a collection mapping variable names to collections of objects containing dimension names. Acceptable such objects are:

    • Symbol: dimension name
    • Type{<:DimensionsionalData.Dimension}: dimension type
    • DimensionsionalData.Dimension: dimension, potentially with indices
    • Nothing: no dimension name provided, dimension name is automatically generated
  • coords: a collection indexable by dimension name specifying the indices of the given dimension. If indices for a dimension in dims are provided, they are used even if the dimension contains its own indices. If a dimension is missing, its indices are automatically generated.

  • kwargs: remaining keywords forwarded to converter functions

InferenceObjects.from_namedtupleFunction
from_namedtuple(posterior::NamedTuple; kwargs...) -> InferenceData
from_namedtuple(posterior::Vector{<:NamedTuple}; kwargs...) -> InferenceData
from_namedtuple(posterior::Matrix{<:NamedTuple}; kwargs...) -> InferenceData
from_namedtuple(posterior::Vector{Vector{<:NamedTuple}}; kwargs...) -> InferenceData
from_namedtuple(
    posterior::NamedTuple,
    sample_stats::Any,
    posterior_predictive::Any,
    predictions::Any,
    log_likelihood::Any;
    kwargs...
) -> InferenceData

Convert a NamedTuple or container of NamedTuples to an InferenceData.

If containers are passed, they are flattened into a single NamedTuple with array elements whose first dimensions correspond to the dimensions of the containers.

Arguments

  • posterior: The data to be converted. It may be of the following types:

    • ::NamedTuple: The keys are the variable names and the values are arrays with dimensions (nchains, ndraws, sizes...).
    • ::Matrix{<:NamedTuple}: Each element is a single draw from a single chain, with array/scalar values with dimensions sizes. The dimensions of the matrix container are (nchains, ndraws)
    • ::Vector{Vector{<:NamedTuple}}: The same as the above case.

Keywords

  • posterior_predictive::Any=nothing: Draws from the posterior predictive distribution
  • sample_stats::Any=nothing: Statistics of the posterior sampling process
  • predictions::Any=nothing: Out-of-sample predictions for the posterior.
  • prior::Any=nothing: Draws from the prior
  • prior_predictive::Any=nothing: Draws from the prior predictive distribution
  • sample_stats_prior::Any=nothing: Statistics of the prior sampling process
  • observed_data::NamedTuple: Observed data on which the posterior is conditional. It should only contain data which is modeled as a random variable. Keys are parameter names and values.
  • constant_data::NamedTuple: Model constants, data included in the model which is not modeled as a random variable. Keys are parameter names and values.
  • predictions_constant_data::NamedTuple: Constants relevant to the model predictions (i.e. new x values in a linear regression).
  • log_likelihood: Pointwise log-likelihood for the data. It is recommended to use this argument as a NamedTuple whose keys are observed variable names and whose values are log likelihood arrays.
  • library: Name of library that generated the draws
  • coords: Map from named dimension to named indices
  • dims: Map from variable name to names of its dimensions

Returns

  • InferenceData: The data with groups corresponding to the provided data

Examples

using InferenceObjects
nchains, ndraws = 2, 10

data1 = (
    x=rand(nchains, ndraws), y=randn(nchains, ndraws, 2), z=randn(nchains, ndraws, 3, 2)
)
idata1 = from_namedtuple(data1)

data2 = [(x=rand(ndraws), y=randn(ndraws, 2), z=randn(ndraws, 3, 2)) for _ in 1:nchains];
idata2 = from_namedtuple(data2)

data3 = [(x=rand(), y=randn(2), z=randn(3, 2)) for _ in 1:nchains, _ in 1:ndraws];
idata3 = from_namedtuple(data3)

data4 = [[(x=rand(), y=randn(2), z=randn(3, 2)) for _ in 1:ndraws] for _ in 1:nchains];
idata4 = from_namedtuple(data4)

General functions

Base.mergeFunction
merge(data::InferenceData, others::InferenceData...) -> InferenceData

Merge InferenceData objects.

The result contains all groups in data and others. If a group appears more than once, the one that occurs first is kept.