Search
  • vinayak sable

Flow of tensors with TensorFlow

Hi Friends, first welcome to my blog on Data Science and Machine Learning. This is my first post on my blog. In this post I am trying to explain a basic idea of TensorFlow. TensorFlow is the second Machine and Deep learning framework of Google. In TensorFlow you can design, build and train deep learning models, also you can numerical computation. For instance, we see code in python. You can also define the architecture on which your code should run on CPU or GPU.

So let's start with Tensorflow. In this post we learn how to create tensor? what is a different type of data type in TensorFlow? and many more... when you start to learn new things then lots of questions come in your mind. so first question at this time is What is Tensor?



Tensor A tensor is a generalization of vectors and matrices and is easily understood as a

multidimensional array. A tensor is an N-dimensional array of data. from Wikipedia, a tensor is an algebraic object that describes a (multilinear) relationship between sets of algebraic objects related to a vector space.

Scalars, Vectors and Matrices If scalars are 0-tensors, then question is that what is 1-tensor? so 1-tensors is a vector i.e list of real numbers. They are either column vectors or row vectors. scalars as 0-tensors, vectors as 1-tensors, and matrices as 2-tensors


So just do some coding in python with Tensorflow. First we try to explain how to install Tensorflow? Choose one of the following TensorFlow packages to install from PyPI

  • TensorFlow —Latest stable release with CPU and GPU support (Ubuntu and Windows).

  • tf-nightly —Preview build (unstable). Ubuntu and Windows include GPU support.

  • tensorflow==1.15 —The final version of TensorFlow 1.x.

Open a new Anaconda/Command Prompt window Once open, type the following on the command line: pip install tensorflow


Import TensorFlow in python using Spyder or Jupyter Notebook

import tensorflow as tf
tf.__version__

# Output
'1.14.0'

x1 = tf.constant([[1,2,3,4]])
print(x1)

# Output
Tensor("Const_3:0", shape=(1, 4), dtype=int32)

Output of about code show only name, shape and dtype of tensor not the value in a tensor. When working with TensorFlow, it’s important to remember that everything is ultimately a graph computation. This means that if you print a TensorFlow operation using Python’s print, it will simply show a description of what that operation is, since no values have been passed through it yet. It will also often show the dimensions that are expected to be in that node, if they’re known. For this we define a session for printing. latter in this post we give more explanation about the session. for printing, we need to define session in TensorFlow.

with tf.Session() as sess:  
    print(x1.eval()) 
# Ouput
[1 2 3 4]

Note for tf.InteractiveSession() -: Difference with a Regular Session is that an on construction InteractiveSession installs itself as the default session. Tensor.eval() and Operation.run() would use that session to run ops.Which permits to use interactive context, like a shell, as it avoids having to pass an explicit Session object to run ops:

tf.InteractiveSession()
<tensorflow.python.client.session.InteractiveSession at 0x2745df90108>
_______________________________________________________________________
a = tf.constant(5.0)
b = tf.constant(6.0)
c = a * b
# We can just use 'c.eval()' without passing 'sess'
print(c.eval())

# Output
30.0

Constant Initialization in Tensorflow

Tensorflow provides a number of functions which instantiate basic tensors in memory. The simplest of these are tf.zeros() and tf.ones(). These tensors are important to live on computer memory in order to be useful to computer programmers. To force the value of the tensor to be returned, we will use the method tf.Tensor.eval() of tensor objects.

t1=tf.zeros(3)
t2 = tf.zeros((2, 3))
t3 = tf.zeros((2,3,5))
print("1D Tesors of Zeros -:\n",t1.eval(),'\n')
print("2D Tesors of Zeros -:\n",t2.eval(),'\n')
print("3D Tesors of Zeros -:\n",t3.eval(),'\n')

# Output
1D Tesors of Zeros -:
 [0. 0. 0.] 

2D Tesors of Zeros -:
 [[0. 0. 0.]
 [0. 0. 0.]] 

3D Tesors of Zeros -:
 [[[0. 0. 0. 0. 0.]
  [0. 0. 0. 0. 0.]
  [0. 0. 0. 0. 0.]]

 [[0. 0. 0. 0. 0.]
  [0. 0. 0. 0. 0.]
  [0. 0. 0. 0. 0.]]] 
 _______________________________________________________________________
 
t=tf.ones(3)
print("1D Tesors of Ones -:",t.eval())

# Output
1D Tesors of Ones -: [1. 1. 1.]

What if we’d like a tensor filled with some quantity besides 0/1? The tf.fill() method provides a nice shortcut for doing so.

b = tf.fill((2, 2), value=70.)
b.eval()

# Output
array([[70., 70.],
       [70., 70.]], dtype=float32)

Random Sampling with Tensors Although working with constant tensors is convenient for testing ideas, it’s much more common to initialize tensors with random values. commonly people are interesting to get sample each entry in tensor from a random distribution. for example Standard Normal Distribution for this we use tf.random_normal


a = tf.random_normal((5, 5), mean=0, stddev=1)
print(a.eval())

# Output
[[-0.20667209 -0.4268163  -1.5726707   0.9915295  -0.79081154]
 [-0.8540901   1.7545588   0.69185066 -1.5782802   1.036152  ]
 [ 0.90504193  0.42189088  0.7592327  -0.32321668  3.4318464 ]
 [-0.8767184   1.1952688  -1.0888201  -0.15835595 -0.805983  ]
 [-0.54621696  0.01953841  1.319754   -1.2828572  -0.39283314]]
 _______________________________________________________________________
a = tf.random_uniform((2, 3,4), minval=-2, maxval=2)
print(a.eval())

# Output
[[[-1.0190034  -1.2483549   0.18776035  1.8042345 ]
  [-0.8656173  -1.7587614   0.46328688 -1.5580826 ]
  [-0.9405389  -0.339499   -1.9089341   1.5258684 ]]

 [[ 1.2225413   0.50671244  1.655313    1.7484803 ]
  [-0.3720379   1.4754715   0.20712471 -1.529345  ]
  [-1.7695241  -1.8245177   0.10762358  1.930368  ]]]

Tensor Operation

TensorFlow involves the manipulation of a tensor. There are four main tensors you can create: tf.Variable, tf.constant, tf.placeholder and tf.SparseTensor.


# Addition of two vectors
c = tf.ones((2, 2))
d = tf.fill((2, 2),3.)
e = c + d
e.eval()

# Output
array([[4., 4.],
       [4., 4.]], dtype=float32)
       
_______________________________________________________________________
# Scaling of Vectors
f = 2 * e
f.eval()

# Output
array([[8., 8.],
       [8., 8.]], dtype=float32)
_______________________________________________________________________
c = tf.fill((2,2), 2.)
d = tf.fill((2,2), 7.)
e = c * d
print(e.eval())

# Output
[[14. 14.]
 [14. 14.]]

Tensors can also be multiplied this way. Note however that this is element-wise multiplication and not matrix multiplication.

Matrix Operations

Machine Learning is just A Matrix Algebra. Most mathematics of machine learning depends on matrix operation. In this session we study matrix operation with TensorFlow. First Starts with Identity Matrix. tf.eye() for fast constructing an identity matrices of a given size.

a = tf.eye(4)
a.eval()

# Output
array([[1., 0., 0., 0.],
       [0., 1., 0., 0.],
       [0., 0., 1., 0.],
       [0., 0., 0., 1.]], dtype=float32)

# Diagonal matrices
r = tf.range(1, 10, 2)
d = tf.diag(r)
d.eval()

# Output
array([[1, 0, 0, 0, 0],
       [0, 3, 0, 0, 0],
       [0, 0, 5, 0, 0],
       [0, 0, 0, 7, 0],
       [0, 0, 0, 0, 9]])

a = tf.ones((2, 3))
at = tf.matrix_transpose(a)
at.eval()

# Output
array([[1., 1.],
       [1., 1.],
       [1., 1.]], dtype=float32)

Matrix Multiplication

Now, let’s suppose we have a pair of matrices we’d like to multiply using matrix multiplication. The easiest way to do so is by invoking tf.matmul().


a = tf.fill((2, 3),5.0)
print(a.eval())
b = tf.random_normal((3, 4))
print(b.eval())

# Output
[[5. 5. 5.]
 [5. 5. 5.]]
 
[[-0.44982427 -0.5109194  -1.0369761  -0.5488375 ]
 [ 0.36409512  1.3481257  -0.95713395  0.12952578]
 [ 0.35181868  0.61409175  0.3076494   0.97709054]]
 
c = tf.matmul(a, b)
c.eval()

# Output
array([[ 2.7792416, -2.1851044, -3.8971133,  1.613812 ],
       [ 2.7792416, -2.1851044, -3.8971133,  1.613812 ]], dtype=float32)

dtype in Tensors

Tensorflow contains variety of dtypes such as tf.float32, tf.float64, tf.int32, tf.int64. For changing its type using casting functions such as functions such as tf.to_double(), tf.to_float(), tf.to_int32(), tf.to_int64() and others.


a = tf.ones((2,2), dtype=tf.int32)
a.eval()

# Output
array([[1, 1],
       [1, 1]])
       
# Convertint to float
b = tf.to_float(a)
b.eval()

# Output
array([[1., 1.],
       [1., 1.]], dtype=float32)

Shape Manipulations of Tensors

Within Tensorflow, tensors are just collections of numbers written in memory. The different shapes are views into the underlying set of numbers that provide different ways of interacting with the set of numbers. At different times, it can be useful to view the same set of numbers as forming tensors with different shapes. tf.reshape() allows tensors to be converted into tensors with different shapes.


a = tf.ones(8)
a.eval()

# Output
array([1., 1., 1., 1., 1., 1., 1., 1.], dtype=float32)

# Reshape vector to matrix
b = tf.reshape(a, (4, 2))
b.eval()

# Output
array([[1., 1.],
       [1., 1.],
       [1., 1.],
       [1., 1.]], dtype=float32)

another function is tf.expand_dims and tf.squeeze. tf.expand_dims adds an extra dimension to a tensor of size 1. It’s useful for increasing the rank of a tensor by one (for example, when converting a vector into a row vector or column vector). tf.squeeze on the other hand removes all dimensions of size 1 from a tensor. It’s a useful way to convert a row or column vector into a flat vector.


a = tf.ones(2)
print(a.get_shape())
a.eval()

# Output
array([1., 1.], dtype=float32)

# axis=0 for row vector or 1 for column vector
b = tf.expand_dims(a, axis=0)
b.eval()

# Output
array([[1., 1.]], dtype=float32)

TensorFlow Variables

The best way to create a variable is by calling the tf.get_variable function. This function requires you to specify the Variable’s name. Variables are manipulated via the tf.Variable class. A tf.Variable represents a tensor whose value can be changed by running ops on it. A tf.Variable exists outside the context of a single session.run call.

a = tf.Variable(tf.ones((2, 2)))
sess = tf.Session()
sess.run(tf.global_variables_initializer())
a.eval(session=sess)

# Output
array([[1., 1.],
       [1., 1.]], dtype=float32)

The evaluation fails since variables have to be explicitly initialized. The easiest way to initialize all variables is to invoketf.global_variables_initializer. Running this operation within a session will initialize all variables in the program


#For updating value of varible
sess.run(a.assign(tf.fill((2,2),6.)))

# Output
array([[6., 6.],
       [6., 6.]], dtype=float32)
       
#Assignment fails when shapes aren’t equal

TensorFlow Placeholders

we have used Variables to manage our data, but there is a more basic structure, the placeholder. A placeholder is simply a variable that we will assign data to at a later date. It allows us to create our operations and build our computation graph, without needing the data. In TensorFlowterminology, we then feed data into the graph through these placeholders.

x = tf.placeholder("float", None)
y = x * 2
with tf.Session() as session:
    result = session.run(y, feed_dict={x: [1, 2, 3]})
    print(result)

# Output
[2. 4. 6.]

Now, when we define the values of x in the feed_dict we can have any number of values. The code should still work, and give the same answer, but now it will also work with any number of values in feed_dict.


with tf.Session() as session:
    result = session.run(y, feed_dict={x: 1})
    print(result)
    
# Output
2.0

Tensorflow Session

It is the main class where we run our TensorFlow operations.

In TensorFlow, a tf.Session() object stores the context under which a computation is performed. we used tf.InteractiveSession() to set up an environment for all TensorFlow computations. This call created a hidden global context for all computations performed. We then used tf.Tensor.eval() to execute our declaratively specified computations. Underneath the hood, this call is evaluated in the context of this hidden global tf.Session. It can be convenient (and often necessary) to use an explicit context for a computation instead of a hidden context

stat = tf.Session()
a = tf.ones((2, 2))
b = tf.matmul(a, a)
print(stat.run(b))
stat.close()

# Output
[[2. 2.]
 [2. 2.]]

Tensorflow Graphs

Tensorflow has a lazy evaluation, in other words Tensorflow will first create a computational graph with the operations as the nodes of the graph and tensors to its edges and the execution happens when the graph executed in a session. This is commonly called a dataflow programming model especially for parallel computing.

a = tf.constant("Hello World")
print(a) 

# Output
Tensor("Const:0", shape=(), dtype=string)

That’s because we haven’t yet run the computation graph in a session so, Tensorflow has still created the graph only.

#first reset Graph
tf.reset_default_graph()

#Let’s create a computation graph to calculate the area of a circle. 
pi = tf.constant(3.14, name="pi")
r = tf.placeholder(tf.float32, name="r")
    
a = r * r * pi
print(a)

# Output
Tensor("mul_7:0", dtype=float32)

Write to events for display graph in tensorboard.

from tensorflow.python.summary.writer.writer import FileWriter
graph = tf.get_default_graph()
print(graph.get_operations())

# Output
[<tf.Operation 'pi' type=Const>, <tf.Operation 'r' type=Placeholder>, <tf.Operation 'mul' type=Mul>, <tf.Operation 'mul_1' type=Mul>]

# Run Session for Computation
with tf.Session() as sess:
    print(a.eval(feed_dict={r: [5]}))

# Output
[78.5]

# write a tf.events in logs/train folder
FileWriter('logs/train', graph=a.graph).close()

Open Tensorbord using Anaconda/Command Prompt

$tensorboard --logdir logs/train

Tensorboard is located in this URL: http://localhost:6006

It could also be located at the following location.



Copy and paste the URL of cmd into your favorite browser. You should see this,


Summary

TensorFlow is an end-to-end open-source platform for machine learning. It has a comprehensive, flexible ecosystem of tools, libraries and community resources that lets researchers push the state-of-the-art in ML and developers easily build and deploy ML-powered applications. The code is available on my GitHub Profile. Please like, comment and share with Friends.


134 views

©2020 by AI Katta.