Operations

This extends the base dynet library with useful operations.

dynn.operations.nll_softmax(logit, y)

This is the same as dy.pickneglogsoftmax.

The main difference is the shorter name and transparent handling of batches. It computes:

\[-\texttt{logit[y]}+\log(\sum_{\texttt{c'}}e^{logit[c']})\]

(softmax then negative log likelihood of y)

Parameters:
dynn.operations.seq_mask(size, lengths, base_val=1, mask_val=0, left_aligned=True)

Returns a mask for a batch sequences of different lengths.

This will return a (size,), len(lengths) shaped expression where the i th element of batch b is base_val iff i<=lengths[b] (and mask_val otherwise).

For example, if size is 4 and lengths is [1,2,4] then the returned mask will be:

(here each row is a batch element)

Parameters:
  • size (int) – Max size of the sequence (must be >=max(lengths))
  • lengths (list) – List of lengths
  • base_val (int, optional) – Value of the mask for non-masked indices (typically 1 for multiplicative masks and 0 for additive masks). Defaults to 1.
  • mask_val (int, optional) – Value of the mask for masked indices (typically 0 for multiplicative masks and -inf for additive masks). Defaults to 0.
  • left_aligned (bool, optional) – Defaults to True.
Returns:

dynet.Expression: Mask expression
dynn.operations.squeeze(x, d=0)

Removes a dimension of size 1 at the given position.

Example:

# (1, 20)
x = dy.zeros((1, 20))
# (20,)
squeeze(x, 0)
# (20, 1)
x = dy.zeros((20, 1))
# (20,)
squeeze(x, 1)
# (20,)
squeeze(x, -1)
dynn.operations.stack(xs, d=0)

Like concatenated but inserts a dimension

d=-1 to insert a dimension at the last position

Parameters:
  • xs (list) – List of expressions with the same dimensions
  • d (int, optional) – Position of the dimension ypu want to insert
dynn.operations.unsqueeze(x, d=0)

Insert a dimension of size 1 at the given position

Example:

# (10, 20)
x = dy.zeros((10, 20))
# (1, 10, 20)
unsqueeze(x, 0)
# (10, 20, 1)
unsqueeze(x, -1)