psiz.keras
psiz.keras.constraints
Module of Keras constraints.
- class psiz.keras.constraints.Center(axis=0)
Constrains the weights to be zero-centered.
This constraint can be used to improve the numerical stability of an embedding.
- get_config()
Return configuration.
- class psiz.keras.constraints.GreaterEqualThan(min_value=0.0)
Constrains the weights to be greater/equal than a value.
- get_config()
Return configuration.
- class psiz.keras.constraints.GreaterThan(min_value=0.0)
Constrains the weights to be greater than a value.
- get_config()
Return configuration.
- class psiz.keras.constraints.LessEqualThan(max_value=0.0)
Constrains the weights to be greater/equal than a value.
- get_config()
Return configuration.
- class psiz.keras.constraints.LessThan(max_value=0.0)
Constrains the weights to be less than a value.
- get_config()
Return configuration.
psiz.keras.initializers
Module of Keras initializers.
psiz.keras.layers
Module of Keras layers.
- class psiz.keras.layers.ALCOVECell(*args, **kwargs)
An RNN-compatible cell implementing ALCOVE.
- build(input_shape)
Build.
- Parameters
input_shape – Expects a dictionary that contains “categorize_stimulus_set” with shape = (batch_size, [1,]).
- call(inputs, states, training=None)
Call.
- Parameters
inputs["stimulus_set"] – The indices of the stimuli. shape=(batch_size, 1)
inputs["objective_query_label"] – One-hot encoding of (objectively correct) query label. shape=(batch_size, n_output)
inputs["gate_weights"] (optional) – Tensor(s) containing gate weights. The actual key value(s) will depend on how the user initialized the layer.
states[0] – A tensor representing batch-specific attention weights that modify the Minkowski distance.
states[1] – A tensor representing batch-specific association weights that map RBF activity to class output activitiy.
- distance(inputs)
Compute distance
- classmethod from_config(config)
Creates a layer from its config.
This method is the reverse of get_config, capable of instantiating the same layer from the config dictionary. It does not handle layer connectivity (handled by Network), nor weights (handled by set_weights).
- Parameters
config – A Python dictionary, typically the output of get_config.
- Returns
A layer instance.
- get_config()
Return layer configuration.
- get_initial_state(inputs=None, batch_size=None, dtype=None)
Get initial state.
- Returns
A list of tensors. The first Tensor represents “attention” weights. The second Tensor represents “association” weights. The attention weights include a sington dimension in anticipation of broadcasting operations in the call method.
- Return type
initial_state
- humble_teacher_loss(y, y_pred)
Humble teacher loss as described in ALCOVE model.
- Parameters
y – shape=(batch_size, units)
y_pred – shape=(batch_size, units)
- Returns
Humble teacher loss.
- class psiz.keras.layers.BraidGate(*args, **kwargs)
A layer that routes inputs to subnetworks.
In a BraidGate the subnetworks are re-combined at the end (in contrast to a BranchGate).
The subnetworks can take a list of inputs, but each subnetwork must output a single tensor. The final output shape must be the same for all subnetworks.
For more information see psiz.keras.layers.Gate
- call(inputs, mask=None)
Call.
- Parameters
inputs – Data Tensors. Can be an n-tuple or single-level dictionary containing Tensors. If n-tuple, the trailing Tensor is assumed to be a “gate weights” Tensor. If a dictionary, one of the fields is assumed to be gate_weights. The tuple format follows [data Tensor, [data Tensor, …], gate_weights Tensor]. The data Tensor(s) follows shape=(batch, m, [n, …]). The gate_weights Tensor follows shape=(batch, g)
mask (optional) – A Tensor indicating which timesteps should be masked.
- compute_output_shape(input_shape)
Computes the output shape of the layer.
- Parameters
input_shape – Shape tuple (tuple of integers) or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- Returns
A tf.TensorShape representing the output shape.
NOTE: This method overrides the TF default, since the default cannot infer the correct output shape.
- class psiz.keras.layers.BranchGate(*args, **kwargs)
A layer that routes inputs to subnetworks.
In a BranchGate the subnetworks are not combined at the end (in contrast to a BraidGate).
For more information see: psiz.keras.layers.Gate
- call(inputs, mask=None)
Call.
- Parameters
inputs – a n-tuple containing a data Tensor and a trailing “gate weights” Tensor. The tuple format follows: [data Tensor, [data Tensor, …], gate_weights Tensor]. The data Tensor(s) follows: shape=(batch, m, [n, …]). the gate_weights Tensor follows: shape=(batch, g)
mask (optional) – A Tensor indicating which timesteps should be masked.
- Returns
A dictionary of Tensors.
- Return type
outputs
- class psiz.keras.layers.CosineSimilarity(*args, **kwargs)
Compute the cosine similarity between pairs of vectors.
Computes the cosine similarity: sum_i w_i u_i v_i / ((sum_i w_i u_i^2)(sum_i w_i v_i^2)), where w is a learnable vector of weights.
Follows the scipy implementation of weighted cosine similarity (scipy.spatial.distance.cosine), except outputs similarity, not distance.
The last axis of the input tensors is consumed in order to compute an inner product (see call method). It is assumed that both input tensors have the same rank, are broadcast-compatible, and have the same size for the last axis.
- build(input_shape)
Build.
- call(inputs)
Call.
- Parameters
inputs – A list of two tf.Tensor’s denoting a the set of vectors to compute pairwise distances. Each tensor is assumed to have the same shape and be at least rank-2. Any additional tensors in the list are ignored. shape = (batch_size, [n, m, …] n_dim)
- Returns
shape = (batch_size, [n, m, …])
- get_config()
Return layer configuration.
- class psiz.keras.layers.DistanceBased(*args, **kwargs)
A distance-based kernel layer.
- build(input_shape)
Build.
- call(inputs)
Call.
Compute k(z_0, z_1), where k is the pairwise kernel function.
- Parameters
inputs – A list of two tf.Tensor’s, plus an optional third Tensor. The first two tensors representing coordinates the pairwise coordinates for which to compute similarity. The coordinate tensors are assumed be at least rank 2, where the first axis indicates the batch size and the last axis indicates the dimensionality of the coordinate space. shape=(batch_size, [n, m, …] n_dim) The optional third tensor is assumed to be rank-2 and indicates group membership. shape=(batch_size, n_col)
- Returns
- A tf.Tensor of pairwise similarites.
shape=(batch_size, [n, m, …])
- Return type
sim_qr
- compute_output_shape(input_shape)
Computes the output shape of the layer.
This method will cause the layer’s state to be built, if that has not happened before. This requires that the layer will later be used with inputs that match the input shape provided here.
- Parameters
input_shape – Shape tuple (tuple of integers) or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- Returns
An input shape tuple.
- classmethod from_config(config)
Create from configuration.
- get_config()
Return layer configuration.
- class psiz.keras.layers.Drop(*args, **kwargs)
A wrapper layer that drops part of the inputs.
Assumues inputs provided to call is a list of Tensors.
- build(input_shape)
Build.
- call(inputs, mask=None)
Call.
- Parameters
inputs – a n-tuple or list containing Tensors.
mask (optional) – A Tensor indicating which timesteps should be masked.
- compute_output_shape(input_shape)
Computes the output shape of the layer.
- Parameters
input_shape – Shape tuple (tuple of integers) or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- Returns
A tf.TensorShape representing the output shape.
NOTE: This method overrides the TF default, since the default cannot infer the correct output shape.
- classmethod from_config(config)
Creates a layer from its config.
This method is the reverse of get_config, capable of instantiating the same layer from the config dictionary. It does not handle layer connectivity (handled by Network), nor weights (handled by set_weights).
- Parameters
config – A Python dictionary, typically the output of get_config.
- Returns
A layer instance.
- get_config()
Return layer configuration.
- class psiz.keras.layers.EmbeddingGammaDiag(*args, **kwargs)
Gamma distribution embedding.
Each embedding point is characterized by a Gamma distribution.
- call(inputs)
Call.
- get_config()
Return layer configuration.
- class psiz.keras.layers.EmbeddingLaplaceDiag(*args, **kwargs)
A distribution-based embedding.
Each embedding point is characterized by a Laplace distribution with a diagonal scale matrix.
- call(inputs)
Call.
- class psiz.keras.layers.EmbeddingLogNormalDiag(*args, **kwargs)
A distribution-based embedding.
Each embedding point is characterized by a Log-Normal distribution with a diagonal scale matrix.
- call(inputs)
Call.
- class psiz.keras.layers.EmbeddingLogitNormalDiag(*args, **kwargs)
A distribution-based embedding.
Each embedding point is characterized by LogitNormal distribution with a diagonal scale matrix.
- call(inputs)
Call.
- class psiz.keras.layers.EmbeddingNormalDiag(*args, **kwargs)
A distribution-based embedding.
Each embedding point is characterized by a Normal distribution with a diagonal scale matrix.
- call(inputs)
Call.
A class for wrapping a shared Embedding.
Build.
Call.
Getter method for embeddings.
Return distribution that creates copies of the source distribution for each stimulus and dimension.
The incoming distribution has, batch_shape=[] event_shape=[1, 1]
We require a distribution with, event_shape=[self.input_dim, self.output_dim].
Create layer from configuration.
This method is the reverse of get_config, capable of instantiating the same layer from the config dictionary.
- Parameters
config – A Python dictionary, typically the output of get_config.
- Returns
A layer instance.
- Return type
layer
Return configuration.
- class psiz.keras.layers.EmbeddingTruncatedNormalDiag(*args, **kwargs)
A distribution-based embedding.
Each embedding point is characterized by a Truncated Normal distribution with a diagonal scale matrix.
- call(inputs)
Call.
- get_config()
Return layer configuration.
- class psiz.keras.layers.EmbeddingVariational(*args, **kwargs)
Variational analog of Embedding layer.
- call(inputs, training=None)
Call.
- property embeddings
Getter method for (posterior) embeddings.
- property input_dim
Getter method for embeddings input_dim.
- property mask_zero
Getter method for embeddings mask_zero.
- property output_dim
Getter method for embeddings output_dim.
- class psiz.keras.layers.ExponentialSimilarity(*args, **kwargs)
Exponential family similarity function.
This exponential-family similarity function is parameterized as: s(x,y) = exp(-beta .* d(x,y).^tau) + gamma, where x and y are n-dimensional vectors. The exponential family function is obtained by integrating across various psychological theories [1,2,3,4].
By default beta=10. and is not trainable to prevent redundancy with trainable embeddings and to prevent short-circuiting any regularizers placed on the embeddings.
References
- [1] Jones, M., Love, B. C., & Maddox, W. T. (2006). Recency
effects as a window to generalization: Separating decisional and perceptual sequential effects in category learning. Journal of Experimental Psychology: Learning, Memory, & Cognition, 32 , 316-332.
- [2] Jones, M., Maddox, W. T., & Love, B. C. (2006). The role of
similarity in generalization. In Proceedings of the 28th annual meeting of the cognitive science society (pp. 405- 410).
- [3] Nosofsky, R. M. (1986). Attention, similarity, and the
identification-categorization relationship. Journal of Experimental Psychology: General, 115, 39-57.
- [4] Shepard, R. N. (1987). Toward a universal law of
generalization for psychological science. Science, 237, 1317-1323.
- call(inputs)
Call.
- Parameters
inputs – A tensor of distances.
- Returns
A tensor of similarities.
- get_config()
Return layer configuration.
- class psiz.keras.layers.Gate(*args, **kwargs)
Abstract layer that routes inputs to subnetworks.
- build(input_shape)
Build.
- classmethod from_config(config)
Creates a layer from its config.
This method is the reverse of get_config, capable of instantiating the same layer from the config dictionary. It does not handle layer connectivity (handled by Network), nor weights (handled by set_weights).
- Parameters
config – A Python dictionary, typically the output of get_config.
- Returns
A layer instance.
- get_config()
Return layer configuration.
- class psiz.keras.layers.GateAdapter(*args, **kwargs)
A layer that adapts inputs for networks with Gates.
- input_keys
List of strings indicating required dictionary keys.
- gating_keys
See __init__ method.
- format_inputs_as_tuple
See __init__ method.
- build(input_shape)
Build.
- call(inputs, training=None, mask=None)
Call.
- Parameters
inputs – A dictionary of Tensors.
training (optional) – see tf.keras.layers.Layer
mask (optional) – see tf.keras.layers.Layer
- get_config()
Get configuration.
- class psiz.keras.layers.GeneralizedInnerProduct(*args, **kwargs)
Compute the inner product between pairs of vectors.
Computes the inner product z_i^T W z_j, where z_i and z_j exist on R^n and W exists on R^n x R^n. By default, no constraints are placed on the matrix W.
The last axis of the input tensors is consumed in order to compute an inner product (see call method). It is assumed that both input tensors have the same rank, are broadcast-compatible, and have the same size for the last axis.
- build(input_shape)
Build.
- call(inputs)
Call.
- Parameters
inputs – A list of two tf.Tensor’s denoting a the set of vectors to compute pairwise distances. Each tensor is assumed to have the same shape and be at least rank-2. Any additional tensors in the list are ignored. shape = (batch_size, [n, m, …] n_dim)
- Returns
shape = (batch_size, [n, m, …])
- get_config()
Return layer configuration.
- class psiz.keras.layers.HeavyTailedSimilarity(*args, **kwargs)
Heavy-tailed family similarity function.
The heavy-tailed similarity function is parameterized as:
s(x,y) = (kappa + (d(x,y).^tau)).^(-alpha),
where x and y are n-dimensional vectors. The heavy-tailed family is a generalization of the Student-t family.
- call(inputs)
Call.
- Parameters
inputs – A tensor of distances.
- Returns
A tensor of similarities.
- get_config()
Return layer configuration.
- class psiz.keras.layers.InnerProduct(*args, **kwargs)
Compute the inner product between pairs of vectors.
Computes the inner product z_i^T W z_j, where z_i and z_j exist on R^n and W exists on R^n x R^n. The constraints placed on the matrix “W” determine the type of the inner product operation. If W is constrained to be the identity matrix, the inner product is a dot product. If W is constrained to be symmetric positive definite, you get a generic inner product.
The layer is implemented using w_tril, the lower triangular Cholesky decomposition of W (W = w_tril @ w_tril^T). This is done because positive definiteness is easier to gaurentee using this parameterization. Internally, a mask is applied to w_tril to ensure the variable w_tril is a lower diagonal matrix.
The last axis of the input tensors is consumed in order to compute an inner product (see call method). It is assumed that both input tensors have the same rank, are broadcast-compatible, and have the same size for the last axis.
- build(input_shape)
Build.
- call(inputs)
Call.
- Parameters
inputs – A list of two tf.Tensor’s denoting a the set of vectors to compute pairwise distances. Each tensor is assumed to have the same shape and be at least rank-2. Any additional tensors in the list are ignored. shape = (batch_size, [n, m, …] n_dim)
- Returns
shape = (batch_size, [n, m, …])
- get_config()
Return layer configuration.
- property w
Return w attribute.
- property w_tril
Return w_tril attribute.
- class psiz.keras.layers.InverseSimilarity(*args, **kwargs)
Inverse-distance similarity function.
The inverse-distance similarity function is parameterized as: s(x,y) = 1 / (d(x,y)**tau + mu), where x and y are n-dimensional vectors.
- call(inputs)
Call.
- Parameters
inputs – A tensor of distances.
- Returns
A tensor of similarities.
- get_config()
Return layer configuration.
- class psiz.keras.layers.Logistic(*args, **kwargs)
A layer for learning a parameterized logistic function.
Inputs are converted via the parameterized logistic function,
f(x) = upper / (1 + exp(-rate*(x - midpoint))))
with the following variable meanings: upper: The upper asymptote of the function’s range. midpoint: The midpoint of the function’s domain and point of maximum growth. rate: The growth rate of the logistic function.
- call(inputs, training=None)
Return logistic function output.
- Parameters
inputs – A tensor of inputs to the logistic function. shape=(batch_size, n, [m, …])
- Returns
- The output of the parameterized logistic function.
shape=(batch_size, n, [m, …])
- Return type
y
- classmethod from_config(config)
Creates a layer from its config.
This method is the reverse of get_config, capable of instantiating the same layer from the config dictionary. It does not handle layer connectivity (handled by Network), nor weights (handled by set_weights).
- Parameters
config – A Python dictionary, typically the output of get_config.
- Returns
A layer instance.
- get_config()
Return layer configuration.
- class psiz.keras.layers.Minkowski(*args, **kwargs)
Minkowski pairwise distance.
- A pairwise Minkowski distance layer that consumes the last axis of
the input tensors (see call method).
NOTE: It is assumed that both tensors have the same rank, are broadcast-compatible, and have the same size for the last axis.
- build(input_shape)
Build.
- call(inputs)
Call.
- Parameters
inputs – A list of two tf.Tensor’s denoting a the set of vectors to compute pairwise distances. Each tensor is assumed to have the same shape and be at least rank-2. Any additional tensors in the list are ignored. shape = (batch_size, [n, m, …] n_dim)
- Returns
shape = (batch_size, [n, m, …])
- get_config()
Return layer configuration.
- class psiz.keras.layers.MinkowskiStochastic(*args, **kwargs)
A stochastic Minkowski distance layer.
- build(input_shape)
Build.
- call(inputs)
Call.
- Parameters
inputs – A list of two tf.Tensor’s denoting a the set of vectors to compute pairwise distance. Each tensor is assumed to have the same shape and be at least rank-2. Any additional tensors in the list are ignored. shape = (batch_size, [n, m, …] n_dim)
- Returns
shape=(batch_size, [n, m, …])
- get_config()
Return layer configuration.
- class psiz.keras.layers.MinkowskiVariational(*args, **kwargs)
Variational analog of Embedding layer.
- call(inputs)
Call.
- Parameters
inputs – A list of two tf.Tensor’s denoting a the set of vectors to compute pairwise distance. Each tensor is assumed to have the same shape and be at least rank-2. Any additional tensors in the list are ignored. shape = (batch_size, [n, m, …] n_dim)
- Returns
shape=(batch_size, [n, m, …])
- property rho
Getter method for (posterior) rho.
- property w
Getter method for (posterior) w.
- class psiz.keras.layers.Proximity(*args, **kwargs)
Abstract base class for pairwise proximity kernel layer.
- A pairwise proximity layer that consumes the last axis of the input
tensors (see call method).
NOTE: It is assumed that both tensors have the same rank, are broadcast-compatible, and have the same size for the last axis.
- call(inputs)
Call.
- classmethod from_config(config)
Create from configuration.
- get_config()
Return layer configuration.
- class psiz.keras.layers.RankSimilarity(*args, **kwargs)
A rank similarity behavior layer.
- call(inputs, training=None)
Return probability of a ranked selection sequence.
- Parameters
inputs –
A dictionary containing the following information (the actual keys will depend on how the user initialized the layer.
- stimulus_set: A tensor containing indices that define
the stimuli used in each trial. shape=(batch_size, max_reference + 1)
- gate_weights (optional): Tensor(s) containing gate
weights.
- Returns
Probability of different behavioral outcomes.
- Return type
outcome_prob
- class psiz.keras.layers.RankSimilarityBase(*args, **kwargs)
A base layer for rank similarity behavior.
- build(input_shape)
Build.
- Parameters
input_shape – Dictionary that should include a key like *_stimulus_set with shape=(batch_size, max_reference + 1).
- classmethod from_config(config)
Creates a layer from its config.
This method is the reverse of get_config, capable of instantiating the same layer from the config dictionary. It does not handle layer connectivity (handled by Network), nor weights (handled by set_weights).
- Parameters
config – A Python dictionary, typically the output of get_config.
- Returns
A layer instance.
- get_config()
Return layer configuration.
- class psiz.keras.layers.RankSimilarityCell(*args, **kwargs)
A stateful rank similarity behavior layer.
- call(inputs, states, training=None)
Return probability of a ranked selection sequence.
- Parameters
inputs[".*_stimulus_set] – A tensor containing indices that define the stimuli used in each trial. shape=(batch_size, max_reference + 1)
- Returns
Probability of different behavioral outcomes.
- Return type
outcome_prob
- NOTE: This computation takes advantage of log-probability
space, exploiting the fact that log(prob=1)=1 to make vectorization cleaner.
- get_initial_state(inputs=None, batch_size=None, dtype=None)
Get initial state.
- class psiz.keras.layers.RateSimilarity(*args, **kwargs)
A rate similarity behavior layer.
- call(inputs, training=None)
Return predicted rating of a trial.
- Parameters
inputs["<data_scope>_stimulus_set"] – A tensor containing indices that define the stimuli used in each trial. shape=(batch_size, n_stimuli_per_trial)
- Returns
- The ratings (on a 0-1 scale) as determined by a
parameterized logistic function.
- Return type
rating
- class psiz.keras.layers.RateSimilarityBase(*args, **kwargs)
A base layer for rate similarity behavior.
Similarities are converted to ratings using a parameterized logistic function,
p(x) = lower + ((upper - lower) / (1 + exp(-rate*(x - midpoint))))
with the following variable meanings: lower: The lower asymptote of the function’s range. upper: The upper asymptote of the function’s range. midpoint: The midpoint of the function’s domain and point of maximum growth. rate: The growth rate of the logistic function.
- build(input_shape)
Build.
- classmethod from_config(config)
Creates a layer from its config.
This method is the reverse of get_config, capable of instantiating the same layer from the config dictionary. It does not handle layer connectivity (handled by Network), nor weights (handled by set_weights).
- Parameters
config – A Python dictionary, typically the output of get_config.
- Returns
A layer instance.
- get_config()
Return layer configuration.
- class psiz.keras.layers.RateSimilarityCell(*args, **kwargs)
A stateful rate behavior layer.
Similarities are converted to probabilities using a parameterized logistic function,
p(x) = lower + ((upper - lower) / (1 + exp(-rate*(x - midpoint))))
with the following variable meanings: lower: The lower asymptote of the function’s range. upper: The upper asymptote of the function’s range. midpoint: The midpoint of the function’s domain and point of maximum growth. rate: The growth rate of the logistic function.
- call(inputs, states, training=None)
Return predicted rating of a trial.
- Parameters
inputs["<data_scope>_stimulus_set"] – A tensor containing indices that define the stimuli used in each trial. shape=(batch_size, n_sample, n_stimuli_per_trial)
- Returns
- The ratings (on a 0-1 scale) as determined by a
parameterized logistic function.
- Return type
rating
- get_initial_state(inputs=None, batch_size=None, dtype=None)
Get initial state.
- class psiz.keras.layers.SoftRank(*args, **kwargs)
A (stateless) soft rank layer.
A (stateless) layer that outputs a soft rank of items based on incoming ‘strength’ associated with each option.
- call(inputs, training=None)
Return probability of a ranked selection sequence.
- Parameters
inputs – A tensor indicating the strengths associated with each option. It is assumed that the last axis indicates the different options.
- Returns
Probability of different behavioral outcomes.
- Return type
outcome_prob
- class psiz.keras.layers.SoftRankBase(*args, **kwargs)
A base layer for ranking options.
A base layer that outputs a soft rank of items based on incoming ‘strength’ associated with each option.
Inputs are assumed to be the ‘strength’ in favor of an option. The probability of selecting a given option is proportional to its strength. The temperature parameter adjust the determinism of the ranking.
The number of options are inferred when the layer is built. Once, built, the soft rank layer can only be used for inputs with the specified number of options.
- build(input_shape)
Build.
- Parameters
input_shape – Shape of strength tensor. shape=(batch_size, [m, n, …] n_option).
- classmethod from_config(config)
Creates a layer from its config.
This method is the reverse of get_config, capable of instantiating the same layer from the config dictionary. It does not handle layer connectivity (handled by Network), nor weights (handled by set_weights).
- Parameters
config – A Python dictionary, typically the output of get_config.
- Returns
A layer instance.
- get_config()
Return layer configuration.
- class psiz.keras.layers.SoftRankCell(*args, **kwargs)
A stateful soft rank layer.
A stateful layer that outputs a soft rank of items based on incoming ‘strength’ associated with each option. The outcome probabilities at the current timestep are biased by past outcome probabilities.
The mixing with past outcomes is determined by the learnable parameter inertia. When inertia is 0.0, there is no bias from past outcomes. As inertia approaches 1.0, the current outcome probabililites are increasingly dominated by the probabilies associated with past outcomes.
p_{t} = (1- inertia) * outcomes_{t} + inertia * p_{t-1}
- call(inputs, states, training=None)
Return probability of a ranked selection sequence.
- Parameters
inputs – A tensor indicating the strengths associated with each option. It is assumed that the last axis indicates the different options.
- Returns
Probability of different behavioral outcomes.
- Return type
outcome_prob
- get_initial_state(inputs=None, batch_size=None, dtype=None)
Get initial state.
- class psiz.keras.layers.SplitGate(*args, **kwargs)
Abstract layer that routes inputs to subnetworks.
- build(input_shape)
Build.
- classmethod from_config(config)
Creates a layer from its config.
This method is the reverse of get_config, capable of instantiating the same layer from the config dictionary. It does not handle layer connectivity (handled by Network), nor weights (handled by set_weights).
- Parameters
config – A Python dictionary, typically the output of get_config.
- Returns
A layer instance.
- get_config()
Return layer configuration.
- class psiz.keras.layers.StochasticEmbedding(*args, **kwargs)
Abstract base class for stochastic embeddings.
Intended to be a drop-in stochastic replacement for tf.keras.layers.Embedding.
- call(inputs)
Call.
- get_config()
Return layer configuration.
- class psiz.keras.layers.StudentsTSimilarity(*args, **kwargs)
Student’s t-distribution similarity function.
The Student’s t-distribution similarity function is parameterized as: s(x,y) = (1 + (((d(x,y)^tau)/alpha))^(-(alpha + 1)/2), where x and y are n-dimensional vectors. The original Student-t kernel proposed by van der Maaten [1] uses a L2 distance (which is governed by the distance kernel), tau=2, and alpha=n_dim-1. By default, all variables are fit to the data.
References
- [1] van der Maaten, L., & Weinberger, K. (2012, Sept).
Stochastic triplet embedding. In Machine learning for signal processing (MLSP), 2012 IEEE international workshop on (p. 1-6). doi:10.1109/MLSP.2012.6349720
- call(inputs)
Call.
- Parameters
inputs – A tensor of distances.
- Returns
A tensor of similarities.
- get_config()
Return layer configuration.
- class psiz.keras.layers.Variational(*args, **kwargs)
An abstract base class for variational layers.
This class can take advantage of a registered KL divergence between the posterior and prior is registered.
- kl_weight
The weighting of the kl term. Should be 1/n_train.
- kl_use_exact
Boolean indicating if a registered KL divergence should be used.
- kl_anneal
An annealing weight that can be accessed using a callback. Iniitalized to one so it has no effect if not used in a callback.
Notes
This layer is not registered as serializable because it is intended to be subclassed. Subclasses must implement call, which should sample from the posterior and call add_kl_loss.
- add_kl_loss(posterior_dist, prior_dist)
Add KL divergence loss.
- build(input_shape)
Build.
- classmethod from_config(config)
Create layer from configuration.
This method is the reverse of get_config, capable of instantiating the same layer from the config dictionary.
- Parameters
config – A Python dictionary, typically the output of get_config.
- Returns
A layer instance.
- Return type
layer
- get_config()
Return configuration.
psiz.keras.models
- class psiz.keras.models.StochasticModel(*args, **kwargs)
An abstract Keras model that accomodates stochastic layers.
Incoming data is transformed by repeating all samples in the batch axis n_sample times for the forward pass. When n_sample is greater than 1, the computed losses and metrics are a better estimate of the expectation. As a side-effect, gradient updates tend to be smoother, reducing the risk of unstable training.
When making predictions, an average across samples is returned.
When calling the model in isolation via the call method, no modifications are made to the inputs.
- n_sample
See init method.
- See `tf.keras.Model` for inherited methods.
- repeat_samples_in_batch_axis()
Transforms data structure by repeating all samples in the batch axis n_sample times.
- average_repeated_samples()
Transforms data structure by averaging over repeated samples.
- disentangle_repeated_samples()
Moves repeated samples to a new axis that has “repeated samples” semantics.
psiz.keras.regularizers
Module of Keras regularizers.
- class psiz.keras.regularizers.AttentionEntropy(rate=0.0)
Entropy-based regularization to encourage sparsity.
- get_config()
Return config.
- class psiz.keras.regularizers.Squeeze(rate=0.0)
Squeeze representation into a low number of dimensions.
Regularizer determines the “max usage” for each dimension by taking the maximum across stimuli. The regularizer places pressure on the representation to only use dimensions if necessary, “squeezing” out dimensions that are not essential for any of the stimuli.
- get_config()
Return config.