It’s well-known at this point that GPT-4/GPT-3.5-turbo is non-deterministic, even at temperature=0.0. This is an odd behavior if you’re used to dense decoder-only models, where temp=0 should imply greedy sampling which should imply full determinism, because the logits for the next token should be a pure function of the input sequence & the model weights.
Despite the dead appearance of this blog, I actually think about it surprisingly often! Over the years, I’ve written a number of draft blogs or summaries that I simply ended up dumped at varying stages of completion.
Five days ago, I published a blog post, describing why TorToiSe could not be fine-tuned.
Today, I have released a fork of DL-Art-School with TorToiSe fine-tuning code. How did that happen?
TorToiSe 🐢 is an open-source Text-To-Speech (TTS) neural network that creates fairly authentic & realistic voices. Checkpoints for local inference have been available since April last year, but its users are seemingly unable to fine-tune the model with additional voice data.
Why is this the case, and how could it be fixed?
On 9th Feb, the National University of Singapore implemented broad restrictions against the use of AI tools to generate work:
