{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"deletable": true,
"editable": true
},
"source": [
"# MATH 382 - Fall 2018\n",
"## Scientific Computing\n",
"Jorge Balbas\n",
"
\n",
"# Lab 2: Linear Algebra with Python\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {
"deletable": true,
"editable": true
},
"source": [
"## Matrix Addition and Subtraction\n",
"\n",
"For two matrices $A$ and $B$ to be added/subtracted, they must be of the same size, and they are added/subtracted element by element $(A \\pm B)_{ij} = A_{ij} \\pm B_{ij}$"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": false,
"deletable": true,
"editable": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"A = [[ 2. 1. -2.]\n",
" [ 1. -3. 4.]] \n",
"\n",
"\n",
"float64 \n",
"\n",
"B = [[-1 1 -2]\n",
" [ 1 4 -1]] \n",
"\n",
"\n",
"int64 \n",
"\n",
"C = [[ 1. 2. -4.]\n",
" [ 2. 1. 3.]] \n",
"\n",
"\n",
"float64 \n",
"\n",
"D = [[ 3. 0. 0.]\n",
" [ 0. -7. 5.]] \n",
"\n",
"\n",
"float64\n"
]
}
],
"source": [
"import numpy as np\n",
"A = np.array([[2, 1, -2], [1, -3, 4]], 'double')\n",
"B = np.array([[-1, 1, -2], [1, 4, -1]])\n",
"print 'A = ', A, '\\n'\n",
"print type(A)\n",
"print A.dtype, '\\n'\n",
"print 'B = ', B, '\\n'\n",
"print type(B)\n",
"print B.dtype, '\\n'\n",
"C = A+B\n",
"print 'C = ', C, '\\n'\n",
"print type(C)\n",
"print C.dtype, '\\n'\n",
"D = A-B\n",
"print 'D = ', D, '\\n'\n",
"print type(C)\n",
"print D.dtype"
]
},
{
"cell_type": "markdown",
"metadata": {
"deletable": true,
"editable": true
},
"source": [
"
\n",
"## Multiplying Matrices and Vectors\n",
"In order to multiply two matrices $A \\in {\\mathbb R}^{m \\times n}$ and $B \\in {\\mathbb R}^{p \\times q}$, $A$ must have as many columns as $B$ has rows ($n = p$), the resulting product is a matrix of size $m \\times q$.\n",
"
\n",
"
\n",
"To define matrix multiplication, we first define Matrix-Vector multiplication\n",
"### Matrix - Vector Multiplication\n",
"The product of the matrix $A \\in {\\mathbb R}^{m \\times n}$ with the vector ${\\bf x} \\in {\\mathbb R}^{n \\times 1}$ is the linear combination of the columns of $A$ with the components of ${\\bf x}$ as coefficients:\n",
"
\n",
"
\n",
"$$ A{\\bf x} = x_1 A_{:,1} + x_2 A_{:,2} + \\dots + x_n A_{:,n} $$\n",
"We can now define Matrix - Matrix multiplication\n",
"\n",
"### Matrix - Matrix Multiplication\n",
"Given matrices $A \\in {\\mathbb R}^{m \\times n}$ and $B \\in {\\mathbb R}^{n \\times p}$, the column $j$ ($j = 1, \\dots, p$) of the product $AB$ is given by the product of $A$ with the $j^{th}$ column of $B$: $(AB)_{:,j} = A B_{:,j}$\n",
"
\n",
"${\\bf Remark:}$ In Python we'll define the matrices $A$ and $B$ as numpy arrays, but the multiplication $A*B$ does not return the matrix product, it returns an array whose entries are element-by-element product "
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": false,
"deletable": true,
"editable": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[[ -2. 1. 4.]\n",
" [ 1. -12. -4.]]\n"
]
}
],
"source": [
"A*B # Array Multiplication\n",
"print(A*B)"
]
},
{
"cell_type": "markdown",
"metadata": {
"deletable": true,
"editable": true
},
"source": [
"To multiply $A$ and $B$ as matirces, we first make sure their dimensions are consistent for multiplying them, and then, we use A.dot(B):"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": false,
"deletable": true,
"editable": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"B = [[-1 1 -2]\n",
" [ 1 4 -1]] \n",
"\n",
"BT = [[-1 1]\n",
" [ 1 4]\n",
" [-2 -1]] \n",
"\n",
"(3, 2) \n",
"\n",
"C = [[ 3. 8.]\n",
" [-12. -15.]] \n",
"\n",
"(2, 2) \n",
"\n"
]
}
],
"source": [
"print 'B = ', B, '\\n'\n",
"BT = B.T # Transpose of matrix B\n",
"print 'BT = ', BT, '\\n'\n",
"print BT.shape, '\\n'\n",
"C = A.dot(BT) # Matrix-Matrix multiplication\n",
"print 'C = ', C, '\\n'\n",
"print C.shape, '\\n'"
]
},
{
"cell_type": "markdown",
"metadata": {
"deletable": true,
"editable": true
},
"source": [
"
\n",
"## Identity Matrices"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": false,
"deletable": true,
"editable": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[[ 1. 0. 0.]\n",
" [ 0. 1. 0.]\n",
" [ 0. 0. 1.]] \n",
"\n",
"[[ 1. 0. 0. 0.]\n",
" [ 0. 1. 0. 0.]\n",
" [ 0. 0. 1. 0.]] \n",
"\n",
"[[ 1. 0. 0. 0.]\n",
" [ 0. 1. 0. 0.]\n",
" [ 0. 0. 1. 0.]\n",
" [ 0. 0. 0. 1.]\n",
" [ 0. 0. 0. 0.]] \n",
"\n"
]
}
],
"source": [
"I3 = np.eye(3)\n",
"print I3, '\\n'\n",
"I34 = np.eye(3,4)\n",
"print I34, '\\n'\n",
"I54 = np.eye(5,4)\n",
"print I54, '\\n'"
]
},
{
"cell_type": "markdown",
"metadata": {
"deletable": true,
"editable": true
},
"source": [
"
\n",
"## Inverse Matrices"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": false,
"deletable": true,
"editable": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[[ 9. -9.]\n",
" [ -9. 26.]] \n",
"\n",
"[[ 1. 0.]\n",
" [ 0. 1.]] \n",
"\n",
"[[ -1.11022302e-16 0.00000000e+00]\n",
" [ 0.00000000e+00 -1.11022302e-16]] \n",
"\n",
"[[ 1. 0.]\n",
" [ 0. 1.]] \n",
"\n",
"[[ -1.11022302e-16 0.00000000e+00]\n",
" [ 0.00000000e+00 -1.11022302e-16]] \n",
"\n"
]
}
],
"source": [
"AAT = A.dot(A.T)\n",
"print AAT, '\\n'\n",
"AATinv = np.linalg.inv(AAT)\n",
"print AAT.dot(AATinv), '\\n'\n",
"print AAT.dot(AATinv) - np.eye(2), '\\n'\n",
"print AATinv.dot(AAT), '\\n'\n",
"print AATinv.dot(AAT) - np.eye(2), '\\n'"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": false,
"deletable": true,
"editable": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[[ 1. -1. 2.]\n",
" [ 2. 0. -1.]\n",
" [ 1. 0. 2.]\n",
" [-2. 3. -2.]] \n",
"\n",
"[[ 1. 2. 1. -2.]\n",
" [-1. 0. 0. 3.]\n",
" [ 2. -1. 2. -2.]] \n",
"\n",
"[[ 10. -7. 6.]\n",
" [ -7. 10. -8.]\n",
" [ 6. -8. 13.]] \n",
"\n",
"335.0 \n",
"\n",
"[[ 0.19701493 0.12835821 -0.0119403 ]\n",
" [ 0.12835821 0.28059701 0.11343284]\n",
" [-0.0119403 0.11343284 0.15223881]] \n",
"\n",
"[[ 1.00000000e+00 4.02455846e-16 -4.87457297e-16]\n",
" [ -1.11022302e-16 1.00000000e+00 -5.55111512e-17]\n",
" [ -1.11022302e-16 2.22044605e-16 1.00000000e+00]] \n",
"\n",
"[[ -2.22044605e-16 4.02455846e-16 -4.87457297e-16]\n",
" [ -1.11022302e-16 0.00000000e+00 -5.55111512e-17]\n",
" [ -1.11022302e-16 2.22044605e-16 0.00000000e+00]] \n",
"\n",
"[[ 1.00000000e+00 1.11022302e-16 0.00000000e+00]\n",
" [ 2.77555756e-16 1.00000000e+00 0.00000000e+00]\n",
" [ -4.23272528e-16 1.66533454e-16 1.00000000e+00]] \n",
"\n",
"[[ -2.22044605e-16 1.11022302e-16 0.00000000e+00]\n",
" [ 2.77555756e-16 0.00000000e+00 0.00000000e+00]\n",
" [ -4.23272528e-16 1.66533454e-16 0.00000000e+00]] \n",
"\n"
]
}
],
"source": [
"C = np.array([[1, -1, 2],[2, 0, -1],[1, 0, 2],[-2, 3, -2]], 'double')\n",
"print C, '\\n'\n",
"CT = C.T\n",
"print CT, '\\n'\n",
"CTC = CT.dot(C)\n",
"print CTC, '\\n'\n",
"print np.linalg.det(CTC), '\\n'\n",
"CTCinv = np.linalg.inv(CTC)\n",
"print CTCinv, '\\n'\n",
"print CTCinv.dot(CTC), '\\n'\n",
"print CTCinv.dot(CTC) - np.eye(3), '\\n'\n",
"print CTC.dot(CTCinv), '\\n'\n",
"print CTC.dot(CTCinv) - np.eye(3), '\\n'"
]
},
{
"cell_type": "markdown",
"metadata": {
"deletable": true,
"editable": true
},
"source": [
"
\n",
"## Norms"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": false,
"deletable": true,
"editable": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[-1 1 -2] \n",
"\n"
]
}
],
"source": [
"x = BT[:,0]\n",
"print x, '\\n'"
]
},
{
"cell_type": "markdown",
"metadata": {
"deletable": true,
"editable": true
},
"source": [
"### Vector $L_p$ Norms"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": false,
"deletable": true,
"editable": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"2.44948974278 \n",
"\n",
"4.0 \n",
"\n",
"2.0 \n",
"\n"
]
}
],
"source": [
"xnorm2 = np.linalg.norm(x)\n",
"print xnorm2, '\\n'\n",
"xnorm1 = np.linalg.norm(x,1)\n",
"print xnorm1, '\\n'\n",
"xnormInf = np.linalg.norm(x,np.inf)\n",
"print xnormInf, '\\n'"
]
},
{
"cell_type": "markdown",
"metadata": {
"deletable": true,
"editable": true
},
"source": [
"### Frobenius Norm of a Matrix"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"collapsed": false,
"deletable": true,
"editable": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"5.9160797831 \n",
"\n"
]
}
],
"source": [
"AnormFrob = np.linalg.norm(A, 'fro')\n",
"print AnormFrob, '\\n'"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": false,
"deletable": true,
"editable": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"5.9160797831 \n",
"\n"
]
}
],
"source": [
"Anorm = np.linalg.norm(A)\n",
"print Anorm, '\\n'"
]
},
{
"cell_type": "markdown",
"metadata": {
"deletable": true,
"editable": true
},
"source": [
"
\n",
"## Eigen Decompositon"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"collapsed": false,
"deletable": true,
"editable": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[[ 10. -7. 6.]\n",
" [ -7. 10. -8.]\n",
" [ 6. -8. 13.]] \n",
"\n"
]
}
],
"source": [
"G = CTC\n",
"print G, '\\n'"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"collapsed": false,
"deletable": true,
"editable": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ 25.13583162 5.39280265 2.47136574] \n",
"\n",
"V = [[-0.51718017 0.70849239 0.48016997]\n",
" [ 0.57449981 -0.12848291 0.80835766]\n",
" [-0.63440889 -0.69392411 0.34057993]] \n",
"\n"
]
}
],
"source": [
"evals, V = np.linalg.eig(G)\n",
"print evals, '\\n'\n",
"print 'V = ', V, '\\n'"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"collapsed": false,
"deletable": true,
"editable": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"1.0 \n",
"\n"
]
}
],
"source": [
"print np.linalg.norm(V[:,0]), '\\n'"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"collapsed": false,
"deletable": true,
"editable": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"335.0 \n",
"\n",
"335.0 \n",
"\n"
]
}
],
"source": [
"print np.linalg.det(G), '\\n'\n",
"print np.prod(evals), '\\n'"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"collapsed": false,
"deletable": true,
"editable": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[[ 10. -7. 6.]\n",
" [ -7. 10. -8.]\n",
" [ 6. -8. 13.]] \n",
"\n",
"[[ -3.55271368e-15 7.10542736e-15 -1.77635684e-15]\n",
" [ -1.77635684e-15 1.77635684e-15 -5.32907052e-15]\n",
" [ 5.32907052e-15 -3.55271368e-15 7.10542736e-15]] \n",
"\n"
]
}
],
"source": [
"Vinv = np.linalg.inv(V)\n",
"print (V.dot(np.diag(evals))).dot(Vinv), '\\n'\n",
"print (V.dot(np.diag(evals))).dot(Vinv) - G, '\\n'"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true,
"deletable": true,
"editable": true
},
"source": [
"## SVD Decomposition"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"collapsed": false,
"deletable": true,
"editable": true
},
"outputs": [
{
"data": {
"text/plain": [
"array([[ 1., -1., 2.],\n",
" [ 2., 0., -1.],\n",
" [ 1., 0., 2.],\n",
" [-2., 3., -2.]])"
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"C"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"collapsed": false,
"deletable": true,
"editable": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[[-0.47082223 -0.2372161 -0.22452874 0.81953755]\n",
" [-0.07977387 0.90899653 -0.39423487 0.10927167]\n",
" [-0.35623315 -0.29254322 -0.73873227 -0.49172253]\n",
" [ 0.80315658 -0.17852812 -0.49843791 0.27317918]] \n",
"\n",
"[[ 5.01356476 0. 0. ]\n",
" [ 0. 2.32224087 0. ]\n",
" [ 0. 0. 1.5720578 ]\n",
" [ 0. 0. 0. ]] \n",
"\n",
"[[-0.51718017 0.57449981 -0.63440889]\n",
" [ 0.70849239 -0.12848291 -0.69392411]\n",
" [-0.48016997 -0.80835766 -0.34057993]] \n",
"\n",
"[[ 1.00000000e+00 -1.00000000e+00 2.00000000e+00]\n",
" [ 2.00000000e+00 1.87267868e-15 -1.00000000e+00]\n",
" [ 1.00000000e+00 1.48746727e-15 2.00000000e+00]\n",
" [ -2.00000000e+00 3.00000000e+00 -2.00000000e+00]] \n",
"\n",
"[[ 1. -1. 2.]\n",
" [ 2. 0. -1.]\n",
" [ 1. 0. 2.]\n",
" [-2. 3. -2.]] \n",
"\n",
"[[ 2.22044605e-16 8.88178420e-16 -1.77635684e-15]\n",
" [ -1.77635684e-15 1.87267868e-15 -1.11022302e-15]\n",
" [ 0.00000000e+00 1.48746727e-15 -1.11022302e-15]\n",
" [ 1.11022302e-15 -4.44089210e-16 6.66133815e-16]] \n",
"\n"
]
}
],
"source": [
"U, d, V = np.linalg.svd(C)\n",
"print U, '\\n'\n",
"D = np.zeros([4,3])\n",
"D[0,0] = d[0]\n",
"D[1,1] = d[1]\n",
"D[2,2] = d[2]\n",
"print D, '\\n'\n",
"print V, '\\n'\n",
"print (U.dot(D)).dot(V), '\\n'\n",
"print C, '\\n'\n",
"print (U.dot(D)).dot(V) - C, '\\n'"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"collapsed": false,
"deletable": true,
"editable": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[[ 6. 0. 5. -9.]\n",
" [ 0. 5. 0. -2.]\n",
" [ 5. 0. 5. -6.]\n",
" [ -9. -2. -6. 17.]] \n",
"\n",
"-9.15933995316e-14\n"
]
}
],
"source": [
"CCT = C.dot(CT)\n",
"print CCT, '\\n'\n",
"print np.linalg.det(CCT)"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ 2.51358316e+01 5.83567557e-16 5.39280265e+00 2.47136574e+00] \n",
"\n",
"[[ 0.47082223 0.81953755 -0.2372161 0.22452874]\n",
" [ 0.07977387 0.10927167 0.90899653 0.39423487]\n",
" [ 0.35623315 -0.49172253 -0.29254322 0.73873227]\n",
" [-0.80315658 0.27317918 -0.17852812 0.49843791]] \n",
"\n"
]
}
],
"source": [
"d2, V2 = np.linalg.eig(CCT)\n",
"print d2, '\\n'\n",
"print V2, '\\n'"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 2",
"language": "python",
"name": "python2"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.13"
}
},
"nbformat": 4,
"nbformat_minor": 1
}