Will Hoover

Week of 5/12 – 5/19/2024

This week has been a lot of time bug diagnosing and fixing, when one issue is fixed another arises. I am getting closer to the training cycle completing, and the more I am troubleshooting this the more comfortable I am getting in the system. This next week the team and I are going to get together to knock out any issues I haven’t been able to solve, my thinking is bringing in some fresh eyes and then explaining it to them, I’m hoping the answer will unlock in either my head or one of my team mates, in any case we should have the model doing text to image generation training soon.

I also spent time this week on improving the training loop by zipping our database up and extending our load functions to incorporate nested file access and zip file access. I started off by examining the way the StyleGAN-T team set up their database to train from and followed the same path to zip my images up to feed them into training, reducing the amount of space that is required from the training dataset. This will allow me to be able to choose in the future if I want to train from a .zip file or a unzipped file. I have been troubleshooting the network, working on getting it to train on the created database that is a combination of the FFHQ dataset and the COCO dataset. I have been making progress and using a much smaller 1000 image dataset sample from the FFHQ dataset to make sure all training is going correctly. Once I get that up and running the training for the text to image generation would just be a time issue after that. To monitor the training I have been looking into TensorBoard so the team can monitor the training while it is happening, I have started to incorporate the code into the network, all that’s left is to troubleshoot any issues that arise and then run the TensorBoard Session while training.

See you next week

-Will