Logging the memory, it seems like it starts the forward pass, memory starts increasing on GPU 0, then OOMs. I wonder if it’s trying to be smart and planning ahead and dequantizing multiple layers at a time. Dequantizing each layer uses ~36 GB of memory so if it was doing this that could cause it to use too much memory. Maybe if we put each layer on alternating GPU’s it could help.
Производитель первого российского аналога лекарства от рака обратился в суд14:57
，推荐阅读有道翻译获取更多信息
The Barbados-born celebrity, whose full name is Robyn Rihanna Fenty, rose to prominence in the early 2000s with hits like Pon de Replay and Umbrella. She recently celebrated 20 years since the release of her first album.，详情可参考手游
FT App on Android & iOS