Contrastive Search might-not-be What You Need
TL;DR
- GPT-NeoX-20B appears to be anisotropic (Isotropy: 0.197)
- Int8 quantisation appears to have ~no effect on isotropy
- I am new to ML so the above could be false, feel free to poke at my findings here
An Informal Reminder
Why Language Models?
A preface to the upcoming series on my attempts to use language models, locally
Recent ML Projects
2 months ago, I was halfway into publishing a series on my TTS project. Today, the site for that project is defunct, and no new posts have been made about it.
What happened?
Multiprocessing and random()
Let’s say, for some odd reason, you’re hosting a Python service that:
- takes requests that run large computations based on a randomly generated number (a “seed”)
- spreads work across multiple workers to handle many requests
Then, it’s likely you’ll introduce a subtle randomness bug that leads to duplicate seeds appearing. Let me explain: