tf.Variable
Create variable
tf.get_variable
- easy to define models which reuse layers
|
|
|
|
Variable collections
collections: name lists of tensors or other objects, such as tf.Variable
instances.
Predefined collections
By default every tf.Variable gets placed in the following two collections:
tf.GraphKeys.GLOBAL_VARIABLES
- variables that can be shared across multiple devices
tf.GraphKeys.TRAINABLE_VARIABLE
- variables for which TensorFlow will calculate gradients
If you don’t want a variable to be trainable, add it to the tf.GraphKeys.LOCAL_VARIABLES
collection instead.
The method is:
or
Custom collections
There is no need to explicitly create a collection.
To add a variable (or any other object) to a collection:
To retrieve a list of all the variables (or other objects) you’ve placed in a collection:
Variable placement
Placement method:
It is particularly important for variables to be in the correct device in distributed settings.
For this reason we provide tf.train.replica_device_setter
, which can automatically place variables in parameter servers.
Initializing variables
In the low-level TensorFlow API, must explicitly initialize the variables by ourselves.
- that is, you are explicitly creating your own graphs and sessions)
Most high-level frameworks automatically initialize variables for you
tf.contrib.slim
,tf.estimator.Estimator
andKeras
Explicit initialization is otherwise useful because it
- allows you not to rerun potentially expensive initializers when reloading a model from a checkpoint ?
- as well as allowing determinism when randomly-initialized variables are shared in a distributed setting.?
tf.global_variables_initializer()
- To initialize all trainable variables
- To be called before training starts
|
|
To initialize variables yourself:
ask which variables have still not been initialized:
Notice
by default tf.global_variables_initializer
does not specify the order in which variables are initialized.
If you use a variable’s value while initializing another variable, use variable.initialized_value()
instead of variable
Using variables
To use the value of a tf.Variable in a TensorFlow graph, simply treat it like a normal tf.Tensor.
To assign a value to a variable, use the methods such as assign
, assign_add
in the tf.Variable
class.
To force a re-read of the value of a variable after something has happened, you can use tf.Variable.read_value.
tf.train.Optimizer
https://www.tensorflow.org/api_docs/python/tf/train/Optimizer
Saving and Restoring
https://www.tensorflow.org/programmers_guide/variables#saving_and_restoring
Sharing variables
https://www.tensorflow.org/programmers_guide/variables#sharing_variables