Will Hoover

Week of 7/7 – 7/14/2024

This week I had to start by fixing a computer issue that came
to be at the end of last week. Just after finishing the integration of
InterFaceGAN into our StyleGAN2 network, my laptop running Pop_OS froze unresponsively.

I was unable to power down the computer, nor was I able to get it started after
unplugging the power cord and allowing the battery to run out. The 500gb hard
drive that was the OS drive was unresponsive, and unmounted. I had to isolate
that drive, format it and reinstall Pop_OS. This took longer than expected due
to the re-installation being broken 4 times during 4 separate installations. I
decided then to use the custom option on Pop_OS which allows formatting of the
hard drives from there, and after a few attempts I was able to get Pop_OS to
install and be stable. Since then, the computer has run fine, and I was able to
get training started.

Our StyleGAN2 model is currently at a point where if it trains
2+ epochs past 140 then it will produce black images, this was true for my
laptop as well. So, I decided then to see if I can get training running on my
desktop. To do that I had to solve the desktop issue that arose a few months
back during our training cycle. I have been working on the network since then,
so I haven’t had the need to look back over the desktop. The issue that was
going on is the desktop was not able to activate the Nvidia drivers for the GPU,
even though they are active and a Nvidia docker container is being run.

I decided to re-install windows to see if that would re-set whatever
issue the computer had with interacting with the Nvidia drivers. This did not
turn out to be a simple task. Even after wiping the OS, getting the docker
container to build was fine, but getting it to run the python file after has
been difficult. I was not able to get it running this week. The train.py file
we use for our StyleGAN2 network, was requesting additional already installed imports
that the container already contained.

Next week since we are getting so close to presenting, we are going to focus on getting the power point ready, finding a image that we can easily reproduce, and work out how we want to present this.