Daniel (Danny) Weitekamp DannyWeitekamp

Georgia Institute of Technology

Recently created

Least recently created

Recently updated

Least recently updated

DannyWeitekamp / Attention.py

Created July 12, 2017 09:25 — forked from cbaziotis/Attention.py

Keras Layer that implements an Attention mechanism for temporal data. Supports Masking. Follows the work of Raffel et al. [https://arxiv.org/abs/1512.08756]

	class Attention(Layer):
	def __init__(self,
	W_regularizer=None, b_regularizer=None,
	W_constraint=None, b_constraint=None,
	bias=True, **kwargs):
	"""
	Keras Layer that implements an Attention mechanism for temporal data.
	Supports Masking.
	Follows the work of Raffel et al. [https://arxiv.org/abs/1512.08756]
	# Input shape

DannyWeitekamp / AttentionWithContext.py

Created July 12, 2017 09:25 — forked from cbaziotis/AttentionWithContext.py

Keras Layer that implements an Attention mechanism, with a context/query vector, for temporal data. Supports Masking. Follows the work of Yang et al. [https://www.cs.cmu.edu/~diyiy/docs/naacl16.pdf] "Hierarchical Attention Networks for Document Classification"

	class AttentionWithContext(Layer):
	"""
	Attention operation, with a context/query vector, for temporal data.
	Supports Masking.
	Follows the work of Yang et al. [https://www.cs.cmu.edu/~diyiy/docs/naacl16.pdf]
	"Hierarchical Attention Networks for Document Classification"
	by using a context vector to assist the attention
	# Input shape
	3D tensor with shape: `(samples, steps, features)`.
	# Output shape