In TensorFlow is there any way to just initialize uninitialised variables?

UPDATE: TensorFlow 0.9 has a new method that "fixes" all this but only if you're using a VariableScope with reuse set to True. tf.report_uninitialized_variables which can be used in one line with sess.run( tf.initialize_variables( list( tf.get_variable(name) for name in sess.run( tf.report_uninitialized_variables( tf.all_variables( ) ) ) ) ) )

or more intelligently through the ability to specify the variables you expect to be initialized:

def guarantee_initialized_variables(session, list_of_variables = None):
    if list_of_variables is None:
        list_of_variables = tf.all_variables()
    uninitialized_variables = list(tf.get_variable(name) for name in
                                   session.run(tf.report_uninitialized_variables(list_of_variables)))
    session.run(tf.initialize_variables(uninitialized_variables))
    return unintialized_variables

This is still less ideal than actually knowing which variables are and are not initialized and taking care of that properly, but in the case of misdirection like the optim classes (see below) it may be hard to avoid.

Also note, tf.initialize_variables cannot evaluate tf.report_uninitialized_variables, so both of them have to be run within the context of the session to work.

---

There is an inelegant but concise way to do it. Before introducing your new variables run temp = set(tf.all_variables()) and afterwards run sess.run(tf.initialize_variables(set(tf.all_variables()) - temp)). These together will only initialize any variables created after the temp value is assigned.

I've been playing with transfer learning, so I wanted a quick way to do it too, but this is the best way I could find. Especially when using things like AdamOptimizer, which doesn't give you easy (or any, I'm not sure) access to the variables it uses. So the following actually shows up in my code. (I initialize the new layer's variables explicitly, and run it once to show the initial error before transfer learning. Just for a sanity check.)

temp = set(tf.all_variables())
train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)
#I honestly don't know how else to initialize ADAM in TensorFlow.
sess.run(tf.initialize_variables(set(tf.all_variables()) - temp))

And it solves all my problems.

EDIT: @Lifu_Huang's answer states the proper way to fix my problem. Theoretically, you should use tf.train.Optimizer.get_slot_names and tf.train.Optimizer.get_slot:

optim = tf.train.AdadeltaOptimizer(1e-4)
loss = cross_entropy(y,yhat)
train_step = optim.minimize(loss)
sess.run(tf.initialize_variables([optim.get_slot(loss, name)
                                  for name in optim.get_slot_names()])

This however gives me AttributeError: 'NoneType' object has no attribute 'initializer'. I'll make edits when I figure out what I did wrong, so you don't make my mistakes.

There is no elegant* way to enumerate the uninitialized variables in a graph. However, if you have access to the new variable objects—let's call them v_6, v_7, and v_8—you can selectively initialize them using tf.initialize_variables():

init_new_vars_op = tf.initialize_variables([v_6, v_7, v_8])
sess.run(init_new_vars_op)

---

* A process of trial and error could be used to identify the uninitialized variables, as follows:

uninitialized_vars = []
for var in tf.all_variables():
    try:
        sess.run(var)
    except tf.errors.FailedPreconditionError:
        uninitialized_vars.append(var)

init_new_vars_op = tf.initialize_variables(uninitialized_vars)
# ...

...however, I would not condone such behavior :-).