phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-07-13 00:26:10 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
llama-stack-mirror/llama_stack/providers
History
Botao Chen 52a21ce78f
Free up memory after post training finishes (#770) 
## context 
Currently, the GPU memory will be continuously occupied after the
training finishes. In this PR, we explicitly delete the reference and
clean up the memory after training finishes.

## test
Before the change, after training a llama 3.2 3B model, >6GB GPU memory
is still occupied

After the change, after training a llama 3.2 3B model, the GPU memory
drops to ~1GB

<img width="156" alt="Screenshot 2025-01-14 at 6 05 17 PM"
src="https://github.com/user-attachments/assets/45d212b1-a651-49f3-aad9-1c0a27fcebcf"
/>
2025-01-14 19:19:38 -08:00
..
inline Free up memory after post training finishes (#770) 2025-01-14 19:19:38 -08:00
registry agents to use tools api (#673) 2025-01-08 19:01:00 -08:00
remote Update spec 2025-01-13 23:16:53 -08:00
tests Update spec 2025-01-13 23:16:53 -08:00
utils [post training] define llama stack post training dataset format (#717) 2025-01-14 12:48:49 -08:00
__init__.py API Updates (#73) 2024-09-17 19:51:35 -07:00
datatypes.py Tools API with brave and MCP providers (#639) 2024-12-19 21:25:17 -08:00