本文档为TensorFlow参考文档，本转载已得到TensorFlow中文社区授权。
Note: Functions taking Tensor arguments can also take anything accepted by tf.convert_to_tensor.
TensorFlow provides a placeholder operation that must be fed with data on execution. For more info, see the section on Feeding data.
Inserts a placeholder for a tensor that will be always fed.
Important: This tensor will produce an error if evaluated. Its value must be fed using the feed_dictoptional argument to Session.run(), Tensor.eval(), or Operation.run().
For example:
x = tf.placeholder(float, shape=(1024, 1024)) y = tf.matmul(x, x) with tf.Session() as sess: print sess.run(y) # ERROR: will fail because x was not fed. rand_array = np.random.rand(1024, 1024) print sess.run(y, feed_dict={x: rand_array}) # Will succeed.A Tensor that may be used as a handle for feeding a value, but not evaluated directly.
TensorFlow provides a set of Reader classes for reading data formats. For more information on inputs and readers, see Reading data.
Base class for different Reader types, that produce a record every step.
Conceptually, Readers convert string 'work units' into records (key, value pairs). Typically the 'work units' are filenames and the records are extracted from the contents of those files. We want a single record produced per step, but a work unit can correspond to many records.
Therefore we introduce some decoupling using a queue. The queue contains the work units and the Reader dequeues from the queue when it is asked to produce a record (via Read()) but it has finished the last work unit.
Creates a new ReaderBase.
Returns the number of records this reader has produced.
This is the same as the number of Read executions that have succeeded.
An int64 Tensor.
Returns the number of work units this reader has finished processing.
An int64 Tensor.
Returns the next record (key, value pair) produced by a reader.
Will dequeue a work unit from queue if necessary (e.g. when the Reader needs to start reading from a new file since it has finished with the previous file).
A tuple of Tensors (key, value).
key: A string scalar Tensor. value: A string scalar Tensor.Op that implements the reader.
Restore a reader to its initial clean state.
The created Operation.
Restore a reader to a previously saved state.
Not all Readers support being restored, so this can produce an Unimplemented error.
The created Operation.
Produce a string tensor that encodes the state of a reader.
Not all Readers support being serialized, so this can produce an Unimplemented error.
A string Tensor.
Whether the Reader implementation can serialize its state.
A Reader that outputs the lines of a file delimited by newlines.
Newlines are stripped from the output. See ReaderBase for supported methods.
Create a TextLineReader.
Returns the number of records this reader has produced.
This is the same as the number of Read executions that have succeeded.
An int64 Tensor.
Returns the number of work units this reader has finished processing.
An int64 Tensor.
Returns the next record (key, value pair) produced by a reader.
Will dequeue a work unit from queue if necessary (e.g. when the Reader needs to start reading from a new file since it has finished with the previous file).
A tuple of Tensors (key, value).
key: A string scalar Tensor. value: A string scalar Tensor.Op that implements the reader.
Restore a reader to its initial clean state.
The created Operation.
Restore a reader to a previously saved state.
Not all Readers support being restored, so this can produce an Unimplemented error.
The created Operation.
Produce a string tensor that encodes the state of a reader.
Not all Readers support being serialized, so this can produce an Unimplemented error.
A string Tensor.
Whether the Reader implementation can serialize its state.
A Reader that outputs the entire contents of a file as a value.
To use, enqueue filenames in a Queue. The output of Read will be a filename (key) and the contents of that file (value).
See ReaderBase for supported methods.
Create a WholeFileReader.
Returns the number of records this reader has produced.
This is the same as the number of Read executions that have succeeded.
An int64 Tensor.
Returns the number of work units this reader has finished processing.
An int64 Tensor.
Returns the next record (key, value pair) produced by a reader.
Will dequeue a work unit from queue if necessary (e.g. when the Reader needs to start reading from a new file since it has finished with the previous file).
A tuple of Tensors (key, value).
key: A string scalar Tensor. value: A string scalar Tensor.Op that implements the reader.
Restore a reader to its initial clean state.
The created Operation.
Restore a reader to a previously saved state.
Not all Readers support being restored, so this can produce an Unimplemented error.
The created Operation.
Produce a string tensor that encodes the state of a reader.
Not all Readers support being serialized, so this can produce an Unimplemented error.
A string Tensor.
Whether the Reader implementation can serialize its state.
A Reader that outputs the queued work as both the key and value.
To use, enqueue strings in a Queue. Read will take the front work string and output (work, work).
See ReaderBase for supported methods.
Create a IdentityReader.
Returns the number of records this reader has produced.
This is the same as the number of Read executions that have succeeded.
An int64 Tensor.
Returns the number of work units this reader has finished processing.
An int64 Tensor.
Returns the next record (key, value pair) produced by a reader.
Will dequeue a work unit from queue if necessary (e.g. when the Reader needs to start reading from a new file since it has finished with the previous file).
A tuple of Tensors (key, value).
key: A string scalar Tensor. value: A string scalar Tensor.Op that implements the reader.
Restore a reader to its initial clean state.
The created Operation.
Restore a reader to a previously saved state.
Not all Readers support being restored, so this can produce an Unimplemented error.
The created Operation.
Produce a string tensor that encodes the state of a reader.
Not all Readers support being serialized, so this can produce an Unimplemented error.
A string Tensor.
Whether the Reader implementation can serialize its state.
A Reader that outputs the records from a TFRecords file.
See ReaderBase for supported methods.
Create a TFRecordReader.
Returns the number of records this reader has produced.
This is the same as the number of Read executions that have succeeded.
An int64 Tensor.
Returns the number of work units this reader has finished processing.
An int64 Tensor.
Returns the next record (key, value pair) produced by a reader.
Will dequeue a work unit from queue if necessary (e.g. when the Reader needs to start reading from a new file since it has finished with the previous file).
A tuple of Tensors (key, value).
key: A string scalar Tensor. value: A string scalar Tensor.Op that implements the reader.
Restore a reader to its initial clean state.
The created Operation.
Restore a reader to a previously saved state.
Not all Readers support being restored, so this can produce an Unimplemented error.
The created Operation.
Produce a string tensor that encodes the state of a reader.
Not all Readers support being serialized, so this can produce an Unimplemented error.
A string Tensor.
Whether the Reader implementation can serialize its state.
A Reader that outputs fixed-length records from a file.
See ReaderBase for supported methods.
Create a FixedLengthRecordReader.
Returns the number of records this reader has produced.
This is the same as the number of Read executions that have succeeded.
An int64 Tensor.
Returns the number of work units this reader has finished processing.
An int64 Tensor.
Returns the next record (key, value pair) produced by a reader.
Will dequeue a work unit from queue if necessary (e.g. when the Reader needs to start reading from a new file since it has finished with the previous file).
A tuple of Tensors (key, value).
key: A string scalar Tensor. value: A string scalar Tensor.Op that implements the reader.
Restore a reader to its initial clean state.
The created Operation.
Restore a reader to a previously saved state.
Not all Readers support being restored, so this can produce an Unimplemented error.
The created Operation.
Produce a string tensor that encodes the state of a reader.
Not all Readers support being serialized, so this can produce an Unimplemented error.
A string Tensor.
Whether the Reader implementation can serialize its state.
TensorFlow provides several operations that you can use to convert various data formats into tensors.
Convert CSV records to tensors. Each column maps to one tensor.
RFC 4180 format is expected for the CSV records. (https://tools.ietf.org/html/rfc4180) Note that we allow leading and trailing spaces with int or float field.
A list of Tensor objects. Has the same type as record_defaults. Each tensor will have the same shape as records.
Reinterpret the bytes of a string as a vector of numbers.
A Tensor of type out_type. A Tensor with one more dimension than the input bytes. The added dimension will have size equal to the length of the elements of bytes divided by the number of bytes to represent out_type.
TensorFlow's recommended format for training examples is serialized Example protocol buffers, described here. They contain Features, described here.
Parses Example protos.
Parses a number of serialized [Example] (https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/core/example/example.proto) protos given in serialized.
names may contain descriptive names for the corresponding serialized protos. These may be useful for debugging purposes, but they have no effect on the output. If not None, names must be the same length as serialized.
This op parses serialized examples into a dictionary mapping keys to Tensor and SparseTensor objects respectively, depending on whether the keys appear in sparse_keys or dense_keys.
The key dense_keys[j] is mapped to a Tensor of type dense_types[j] and of shape (serialized.size(),) + dense_shapes[j].
dense_defaults provides defaults for values referenced using dense_keys. If a key is not present in this dictionary, the corresponding dense Feature is required in all elements of serialized.
dense_shapes[j] provides the shape of each Feature entry referenced by dense_keys[j]. The number of elements in the Feature corresponding to dense_key[j] must always have np.prod(dense_shapes[j]) entries. The returned Tensor for dense_key[j] has shape [N] + dense_shape[j], where N is the number of Examples in serialized.
The key sparse_keys[j] is mapped to a SparseTensor of type sparse_types[j]. The SparseTensorrepresents a ragged matrix. Its indices are [batch, index] where batch is the batch entry the value is from, and index is the value's index in the list of values associated with that feature and example.
Examples:
For example, if one expects a tf.float32 sparse feature ft and three serialized Examples are provided:
serialized = [ features: { feature: [ key: { "ft" value: float_list: { value: [1.0, 2.0] } } ] }, features: { feature: [] }, features: { feature: [ key: { "ft" value: float_list: { value: [3.0] } } ] } ]then the output will look like:
{"ft": SparseTensor(indices=[[0, 0], [0, 1], [2, 0]], values=[1.0, 2.0, 3.0], shape=(3, 2)) }Given two Example input protos in serialized:
[ features: { feature: { key: "kw" value: { bytes_list: { value: [ "knit", "big" ] } } } feature: { key: "gps" value: { float_list: { value: [] } } } }, features: { feature: { key: "kw" value: { bytes_list: { value: [ "emmy" ] } } } feature: { key: "dank" value: { int64_list: { value: [ 42 ] } } } feature: { key: "gps" value: { } } } ]And arguments
names: ["input0", "input1"], sparse_keys: ["kw", "dank", "gps"] sparse_types: [DT_STRING, DT_INT64, DT_FLOAT]Then the output is a dictionary:
{ "kw": SparseTensor( indices=[[0, 0], [0, 1], [1, 0]], values=["knit", "big", "emmy"] shape=[2, 2]), "dank": SparseTensor( indices=[[1, 0]], values=[42], shape=[2, 1]), "gps": SparseTensor( indices=[], values=[], shape=[2, 0]), }For dense results in two serialized Examples:
[ features: { feature: { key: "age" value: { int64_list: { value: [ 0 ] } } } feature: { key: "gender" value: { bytes_list: { value: [ "f" ] } } } }, features: { feature: { key: "age" value: { int64_list: { value: [] } } } feature: { key: "gender" value: { bytes_list: { value: [ "f" ] } } } } ]We can use arguments:
names: ["input0", "input1"], dense_keys: np.array(["age", "gender"]), dense_types: [tf.int64, tf.string], dense_defaults: { "age": -1 # "age" defaults to -1 if missing # "gender" has no specified default so it's required } dense_shapes: [(1,), (1,)], # age, gender, label, weightAnd the expected output is:
{ "age": [[0], [-1]], "gender": [["f"], ["f"]], }A dict mapping keys to Tensors and SparseTensors.
Parses a single Example proto.
Similar to parse_example, except:
For dense tensors, the returned Tensor is identical to the output of parse_example, except there is no batch dimension, the output shape is the same as the shape given in dense_shape.
For SparseTensors, the first (batch) column of the indices matrix is removed (the indices matrix is a column vector), the values vector is unchanged, and the first (batch_size) entry of the shape vector is removed (it is now a single element vector).
See also parse_example.
A dictionary mapping keys to Tensors and SparseTensors.
TensorFlow provides several implementations of 'Queues', which are structures within the TensorFlow computation graph to stage pipelines of tensors together. The following describe the basic Queue interface and some implementations. To see an example use, see Threading and Queues.
Base class for queue implementations.
A queue is a TensorFlow data structure that stores tensors across multiple steps, and exposes operations that enqueue and dequeue tensors.
Each queue element is a tuple of one or more tensors, where each tuple component has a static dtype, and may have a static shape. The queue implementations support versions of enqueue and dequeue that handle single elements, versions that support enqueuing and dequeuing a batch of elements at once.
See tf.FIFOQueue and tf.RandomShuffleQueue for concrete implementations of this class, and instructions on how to create them.
Enqueues one element to this queue.
If the queue is full when this operation executes, it will block until the element has been enqueued.
The operation that enqueues a new tuple of tensors to the queue.
Enqueues zero or elements to this queue.
This operation slices each component tensor along the 0th dimension to make multiple queue elements. All of the tensors in vals must have the same size in the 0th dimension.
If the queue is full when this operation executes, it will block until all of the elements have been enqueued.
The operation that enqueues a batch of tuples of tensors to the queue.
Dequeues one element from this queue.
If the queue is empty when this operation executes, it will block until there is an element to dequeue.
The tuple of tensors that was dequeued.
Dequeues and concatenates n elements from this queue.
This operation concatenates queue-element component tensors along the 0th dimension to make a single component tensor. All of the components in the dequeued tuple will have size n in the 0th dimension.
If the queue contains fewer than n elements when this operation executes, it will block until nelements have been dequeued.
The tuple of concatenated tensors that was dequeued.
Compute the number of elements in this queue.
A scalar tensor containing the number of elements in this queue.
Closes this queue.
This operation signals that no more elements will be enqueued in the given queue. Subsequent enqueueand enqueue_many operations will fail. Subsequent dequeue and dequeue_many operations will continue to succeed if sufficient elements remain in the queue. Subsequent dequeue and dequeue_manyoperations that would block will fail immediately.
If cancel_pending_enqueues is True, all pending requests will also be cancelled.
The operation that closes the queue.
Constructs a queue object from a queue reference.
The list of dtypes for each component of a queue element.
The name of the underlying queue.
The underlying queue reference.
A queue implementation that dequeues elements in first-in-first out order.
See tf.QueueBase for a description of the methods on this class.
Creates a queue that dequeues elements in a first-in first-out order.
A FIFOQueue has bounded capacity; supports multiple concurrent producers and consumers; and provides exactly-once delivery.
A FIFOQueue holds a list of up to capacity elements. Each element is a fixed-length tuple of tensors whose dtypes are described by dtypes, and whose shapes are optionally described by the shapesargument.
If the shapes argument is specified, each component of a queue element must have the respective fixed shape. If it is unspecified, different queue elements may have different shapes, but the use of dequeue_many is disallowed.
A queue implementation that dequeues elements in a random order.
See tf.QueueBase for a description of the methods on this class.
Create a queue that dequeues elements in a random order.
A RandomShuffleQueue has bounded capacity; supports multiple concurrent producers and consumers; and provides exactly-once delivery.
A RandomShuffleQueue holds a list of up to capacity elements. Each element is a fixed-length tuple of tensors whose dtypes are described by dtypes, and whose shapes are optionally described by the shapes argument.
If the shapes argument is specified, each component of a queue element must have the respective fixed shape. If it is unspecified, different queue elements may have different shapes, but the use of dequeue_many is disallowed.
The min_after_dequeue argument allows the caller to specify a minimum number of elements that will remain in the queue after a dequeue or dequeue_many operation completes, to ensure a minimum level of mixing of elements. This invariant is maintained by blocking those operations until sufficient elements have been enqueued. The min_after_dequeue argument is ignored after the queue has been closed.
Returns the set of files matching a pattern.
Note that this routine only supports wildcard characters in the basename portion of the pattern, not in the directory portion.
A Tensor of type string. A vector of matching filenames.
Reads and outputs the entire contents of the input filename.
A Tensor of type string.
TensorFlow functions for setting up an input-prefetching pipeline. Please see the reading data how-to for context.
The "producer" functions add a queue to the graph and a corresponding QueueRunner for running the subgraph that fills that queue.
Save the list of files matching pattern, so it is only computed once.
A variable that is initialized to the list of files matching pattern.
Returns tensor num_epochs times and then raises an OutOfRange error.
tensor or OutOfRange.
Produces the integers from 0 to limit-1 in a queue.
A Queue with the output integers. A QueueRunner for the Queue is added to the current Graph's QUEUE_RUNNER collection.
Produces a slice of each Tensor in tensor_list.
Implemented using a Queue -- a QueueRunner for the Queue is added to the current Graph's QUEUE_RUNNER collection.
A list of tensors, one for each element of tensor_list. If the tensor in tensor_list has shape [N, a, b, .., z], then the corresponding output tensor will have shape [a, b, ..., z].
Output strings (e.g. filenames) to a queue for an input pipeline.
A queue with the output strings. A QueueRunner for the Queue is added to the current Graph's QUEUE_RUNNER collection.
These functions add a queue to the graph to assemble a batch of examples, with possible shuffling. They also add a QueueRunner for running the subgraph that fills that queue.
Use batch or batch_join for batching examples that have already been well shuffled. Use shuffle_batch orshuffle_batch_join for examples that would benefit from additional shuffling.
Use batch or shuffle_batch if you want a single thread producing examples to batch, or if you have a single subgraph producing examples but you want to run it in N threads (where you increase N until it can keep the queue full). Use batch_join or shuffle_batch_join if you have N different subgraphs producing examples to batch and you want them run by N threads.
Creates batches of tensors in tensor_list.
This function is implemented using a queue. A QueueRunner for the queue is added to the current Graph's QUEUE_RUNNER collection.
If enqueue_many is False, tensor_list is assumed to represent a single example. An input tensor with shape [x, y, z] will be output as a tensor with shape [batch_size, x, y, z].
If enqueue_many is True, tensor_list is assumed to represent a batch of examples, where the first dimension is indexed by example, and all members of tensor_list should have the same size in the first dimension. If an input tensor has shape [*, x, y, z], the output will have shape [batch_size, x, y, z]. The capacity argument controls the how long the prefetching is allowed to grow the queues.
A list of tensors with the same number and types as tensor_list.
Runs a list of tensors to fill a queue to create batches of examples.
Enqueues a different list of tensors in different threads. Implemented using a queue -- a QueueRunner for the queue is added to the current Graph's QUEUE_RUNNER collection.
len(tensor_list_list) threads will be started, with thread i enqueuing the tensors fromtensor_list_list[i]. tensor_list_list[i1][j] must match tensor_list_list[i2][j] in type and shape, except in the first dimension if enqueue_many is true.
If enqueue_many is False, each tensor_list_list[i] is assumed to represent a single example. An input tensor x will be output as a tensor with shape [batch_size] + x.shape.
If enqueue_many is True, tensor_list_list[i] is assumed to represent a batch of examples, where the first dimension is indexed by example, and all members of tensor_list_list[i] should have the same size in the first dimension. The slices of any input tensor x are treated as examples, and the output tensors will have shape [batch_size] + x.shape[1:].
The capacity argument controls the how long the prefetching is allowed to grow the queues.
A list of tensors with the same number and types as tensor_list_list[i].
Creates batches by randomly shuffling tensors.
This function adds the following to the current Graph:
A shuffling queue into which tensors from tensor_list are enqueued.A dequeue_many operation to create batches from the queue.A QueueRunner to QUEUE_RUNNER collection, to enqueue the tensors from tensor_list.If enqueue_many is False, tensor_list is assumed to represent a single example. An input tensor with shape [x, y, z] will be output as a tensor with shape [batch_size, x, y, z].
If enqueue_many is True, tensor_list is assumed to represent a batch of examples, where the first dimension is indexed by example, and all members of tensor_list should have the same size in the first dimension. If an input tensor has shape [*, x, y, z], the output will have shape [batch_size, x, y, z].
The capacity argument controls the how long the prefetching is allowed to grow the queues.
For example:
# Creates batches of 32 images and 32 labels. image_batch, label_batch = tf.train.shuffle_batch( [single_image, single_label], batch_size=32, num_threads=4, capacity=50000, min_after_dequeue=10000)A list of tensors with the same number and types as tensor_list.
Create batches by randomly shuffling tensors.
This version enqueues a different list of tensors in different threads. It adds the following to the current Graph:
A shuffling queue into which tensors from tensor_list_list are enqueued.A dequeue_many operation to create batches from the queue.A QueueRunner to QUEUE_RUNNER collection, to enqueue the tensors from tensor_list_list.len(tensor_list_list) threads will be started, with thread i enqueuing the tensors from tensor_list_list[i]. tensor_list_list[i1][j] must match tensor_list_list[i2][j] in type and shape, except in the first dimension if enqueue_many is true.
If enqueue_many is False, each tensor_list_list[i] is assumed to represent a single example. An input tensor with shape [x, y, z] will be output as a tensor with shape [batch_size, x, y, z].
If enqueue_many is True, tensor_list_list[i] is assumed to represent a batch of examples, where the first dimension is indexed by example, and all members of tensor_list_list[i] should have the same size in the first dimension. If an input tensor has shape [*, x, y, z], the output will have shape [batch_size, x, y, z].
The capacity argument controls the how long the prefetching is allowed to grow the queues.
A list of tensors with the same number and types as tensor_list_list[i].
相关资源：神奇脑波v6.3.8.apk