Skip to content

Instantly share code, notes, and snippets.

@beomjunshin-ben

beomjunshin-ben/Tensor&Matrix Operation Cookbook.ipynb

Created September 28, 2015 08:43
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save beomjunshin-ben/8e0200b26a2814fb7624 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save beomjunshin-ben/8e0200b26a2814fb7624 to your computer and use it in GitHub Desktop.
Download ZIP
Display the source blob
Display the rendered blob
Raw
Tensor&Matrix Operation Cookbook.ipynb
{
"cells": [
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"import numpy as np"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### TASK #1 (Matrix-Matrix=>Tensor)\n",
"\n",
"즉, 모든 X 각각에 대해 c1, c2, .. cH를 뺀 Tensor 를 얻고자 함\n",
"\n",
"```\n",
"X: (N, I)\n",
"Center: (H, I)\n",
"```\n",
"=>`X - Center: (N, H, I)` \n",
"\n",
"Braodcasting & reshaping\n",
"\n",
"```\n",
"X_reshaped (3d array): N x 1 x I\n",
"Center (2d array): H x I\n",
"Result (3d array): N x H x I\n",
"```\n",
"\n",
"Example Case N = 5, H = 3, I = 2"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"array([[1, 1],\n",
" [2, 2],\n",
" [3, 3],\n",
" [4, 4],\n",
" [5, 5]])"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"X = np.array(\n",
" [\n",
" [1, 1],\n",
" [2, 2],\n",
" [3, 3],\n",
" [4, 4],\n",
" [5, 5]\n",
" ]\n",
")\n",
"X"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"array([[2, 2],\n",
" [2, 2],\n",
" [2, 2]])"
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"Center = np.array(\n",
" [\n",
" [2, 2],\n",
" [2, 2],\n",
" [2, 2]\n",
" ]\n",
")\n",
"Center"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(5, 2) (3, 2)\n"
]
}
],
"source": [
"print(X.shape, Center.shape)"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(5, 3, 2)\n",
"[[[-1 -1]\n",
" [-1 -1]\n",
" [-1 -1]]\n",
"\n",
" [[ 0 0]\n",
" [ 0 0]\n",
" [ 0 0]]\n",
"\n",
" [[ 1 1]\n",
" [ 1 1]\n",
" [ 1 1]]\n",
"\n",
" [[ 2 2]\n",
" [ 2 2]\n",
" [ 2 2]]\n",
"\n",
" [[ 3 3]\n",
" [ 3 3]\n",
" [ 3 3]]]\n"
]
}
],
"source": [
"# reshaping & broadcasting\n",
"Result = X.reshape((5,1,2)) - Center\n",
"print(Result.shape)\n",
"print(Result)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### TASK #2 (Matrix tensordot Tensor => Tensor)\n",
"\n",
"Matrix(아래 예에선 Vector)와 Tensor의 곱으로 Matrix 얻기 ( 뉴럴넷에서 흔히 등장하는 패턴 )\n",
"\n",
"$$\n",
"\\frac {\\partial L} {\\partial Z} \\frac {\\partial Z} {\\partial W} \\\\\n",
"$$\n",
"\n",
"(1, H) tensordot (H, H, I)\n",
"\n",
"즉, 오른쪽 Tensor는 (H, I) 매트릭스가 (H, 1)만큼 있는 벡터인 셈\n",
"\n",
"```\n",
"Delta: (1, H) (매칭될 axis = 1)\n",
"A: (H, H, I) (매칭될 axis = 0) // H, H 일 필요는 없음\n",
"```\n",
"\n",
"Example Case H = 3, I = 2"
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"array([[1, 2, 3]])"
]
},
"execution_count": 28,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"Delta = np.array(\n",
" [\n",
" [1, 2, 3]\n",
" ]\n",
")\n",
"Delta"
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"array([[[2, 4],\n",
" [3, 5],\n",
" [3, 5]],\n",
"\n",
" [[1, 2],\n",
" [3, 4],\n",
" [3, 4]],\n",
"\n",
" [[1, 0],\n",
" [0, 1],\n",
" [0, 1]]])"
]
},
"execution_count": 30,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"A = np.array(\n",
" [\n",
" [\n",
" [2,4],\n",
" [3,5],\n",
" [3,5]\n",
" ],\n",
" [\n",
" [1,2],\n",
" [3,4],\n",
" [3,4]\n",
" ],\n",
" [\n",
" [1,0],\n",
" [0,1],\n",
" [0,1]\n",
" ]\n",
" ]\n",
")\n",
"A"
]
},
{
"cell_type": "code",
"execution_count": 34,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"array([[[ 7, 8],\n",
" [ 9, 16],\n",
" [ 9, 16]]])"
]
},
"execution_count": 34,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Using tensor dot with axis Concept\n",
"np.tensordot(Delta, A, (1, 0))"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.4.3"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment