Discussion about this post

User's avatar
MI's avatar

GPT-3 has 350 MB of weights and will take atleast 8 of the latest 80GB A100's. That will increase cloud cost quite a bit from 1.9$/hour to 19$/hour. The A100's are also twice as fast as V100, so that will offset some of the cost. Overall, I estimate that margin will drop to 10x from 60x

Expand full comment
Ptrenko's avatar

Does the prompt also consume tokens?

I doubt you could put a book and generate its sentiment for the cost of 5-6 tokens right?

Expand full comment
2 more comments...

No posts