When a GPU is available, TensorFlow automatically copies tensors between CPU and GPU memory (see Using GPUs).

When eager execution is enabled, automatic copying between devices is disabled by default. When executing imperatively, copying data between CPU and GPU is more likely to become a performance bottleneck. Avoiding automatic copying makes it easier to identify such bottlenecks. For example, consider the program:

with tf.device(“/cpu:0”):
  x = tf.ones([2, 2])
with tf.device(“/gpu:0”):
  y = tf.matmul(x, x)

This will fail with an error like:

Tensors on conflicting devices: cannot compute MatMul as input #0 was expected to be on /job:localhost/replica:0/task:0/device:GPU:0 but is actually on /job:localhost/replica:0/task:0/device:CPU:0 …

indicating that the matmul operation cannot be conducted on the GPU as its inputs were host memory.

If you run into this situation, your options are:

• Accept the potential performance hit by explicitly enabling automatic copying between devices:

tfe.enable_eager_execution(device_policy=tfe.DEVICE_PLACEMENT_SILENT)

• Explicitly copy the tensor yourself using tf.identity or Tensor.gpu():

with tf.device(“/gpu:0”):
  x = tf.identity(x)
  y = tf.matmul(x, x)