How to combine a pre-trained KerasLayer from TensorFlow (v. 2) H

ghz 2days ago ⋅ 5 views

How to combine a pre-trained KerasLayer from TensorFlow (v. 2) Hub and tfrecords?

I have a tfrecord with 23 classes with 35 images in each class (805 in total). My current tfrecord read function is:

def read_tfrecord(serialized_example):
 feature_description = {
    'image': tf.io.FixedLenFeature((), tf.string),
    'label': tf.io.FixedLenFeature((), tf.int64),
    'height': tf.io.FixedLenFeature((), tf.int64),
    'width': tf.io.FixedLenFeature((), tf.int64),
    'depth': tf.io.FixedLenFeature((), tf.int64)
 }

 example = tf.io.parse_single_example(serialized_example, feature_description)
 image = tf.io.parse_tensor(example['image'], out_type=float)
 image_shape = [example['height'], example['width'], example['depth']]
 image = tf.reshape(image, image_shape)
 label = tf.cast(example["label"], tf.int32)
 image = image/255

 return image, label

I then have a make_dataset function that looks like this:

def make_dataset(tfrecord, BATCH_SIZE, EPOCHS, cache=True):
 files = tf.data.Dataset.list_files(os.path.join(os.getcwd(), tfrecord))
 dataset = tf.data.TFRecordDataset(files)

 if cache:
    if isinstance(cache, str):
      dataset = dataset.cache(cache)
    else:
      dataset = dataset.cache()

 dataset = dataset.shuffle(buffer_size=FLAGS.shuffle_buffer_size)
 dataset = dataset.map(map_func=read_tfrecord, num_parallel_calls=AUTOTUNE)
 dataset = dataset.repeat(EPOCHS)
 dataset = dataset.batch(batch_size=BATCH_SIZE)
 dataset = dataset.prefetch(buffer_size=AUTOTUNE)

 return dataset

This make_dataset function gets passed into

train_ds = make_dataset(tfrecord=FLAGS.tf_record, BATCH_SIZE=BATCH_SIZE, EPOCHS=EPOCH)
image_batch, label_batch = next(iter(train_ds))
feature_extractor_layer = hub.KerasLayer(url, input_shape=IMAGE_SHAPE + (3,)) 
feature_batch = feature_extractor_layer(image_batch)
feature_extractor_layer.trainable = False
model = tf.keras.Sequential([feature_extractor_layer, layers.Dense(2048, input_shape=(2048,)), layers.Dense(len(CLASS_NAMES), activation='softmax')])

model.summary()
predictions = model(image_batch)
model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=rate),
                              loss='categorical_crossentropy',
                              metrics=['acc'])

batch_stats_callback = CollectBatchStats()
STEPS_PER_EPOCH = np.ceil(image_count / BATCH_SIZE)
history = model.fit(image_batch, label_batch, epochs=EPOCH, batch_size=BATCH_SIZE, steps_per_epoch=STEPS_PER_EPOCH, callbacks=[batch_stats_callback])

This code runs in the sense that it outputs the usual information about how many epochs I have and some training accuracy data (which is 0 with a loss around 100k). The error I get doesn't have any meaning to me, as it says: Function instantiation has undefined input shape at index: 100 in the outer inference context. You can substitute the number to anything below 1000 (not sure if it ever surpasses the number of images I have in my tfrecord).

I'm at a complete loss with this one.

EDIT:

It seems this "error" I was getting was nothing but a warning message. I suspect it is related to the use of TensorFlow Hub and potentially eager execution. I added

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'

at the beginning of the file and the warning has vanished.

Answer

It looks like you've made great progress with your code. The issue you mentioned, particularly the warning about "undefined input shape," seems to be related to the input shape of your feature_extractor_layer (which uses TensorFlow Hub) not being fully defined at the point of instantiation.

Here are a few things you could try to address this:

1. Verify the input shape of the feature extractor:

Make sure that the input shape you pass to the KerasLayer corresponds to the shape of your input images. In your code, you define the input shape as IMAGE_SHAPE + (3,), but it’s unclear if IMAGE_SHAPE is set correctly. Ensure that IMAGE_SHAPE is a tuple like (height, width) that matches the dimensions of the images in your TFRecord (without including the channel dimension 3 for RGB). For example, if your images are 128x128, set:

IMAGE_SHAPE = (128, 128)  # Set the correct height and width

and update:

feature_extractor_layer = hub.KerasLayer(url, input_shape=IMAGE_SHAPE + (3,))

2. Check for Eager Execution Issues:

TensorFlow Hub layers sometimes have trouble with static input shapes, especially in the context of eager execution. You can try setting the model input shape explicitly to ensure that the feature extractor knows its shape:

feature_extractor_layer = hub.KerasLayer(url, input_shape=(None, IMAGE_SHAPE[0], IMAGE_SHAPE[1], 3))

This forces TensorFlow to expect images with the specified dimensions but allows for dynamic batch sizes (None).

3. Set up the correct labels format:

Since you're using categorical_crossentropy as the loss function, make sure your labels are in one-hot encoded format. If they aren't, you can convert them like this:

label_batch = tf.one_hot(label_batch, len(CLASS_NAMES))

If your labels are integers (which they seem to be in the TFRecord file), you should one-hot encode them to match the output of the final Dense layer with softmax activation.

4. Ensure your STEPS_PER_EPOCH is set correctly:

In the code snippet you provided, you compute STEPS_PER_EPOCH using image_count:

STEPS_PER_EPOCH = np.ceil(image_count / BATCH_SIZE)

Make sure image_count correctly represents the total number of images in your dataset. You can compute it by reading the total number of records in your TFRecord:

image_count = sum(1 for _ in tf.data.TFRecordDataset(files))

This ensures that the number of steps per epoch is correct.

5. Debugging Tips:

  • To debug, print out the shapes of image_batch and label_batch before passing them to model.fit(): 
    print(image_batch.shape)
print(label_batch.shape)

This can help verify that the input shapes align with what the model expects.

Let me know if you run into further issues or need more clarification!

#py