Commit graph

5 commits

Author SHA1 Message Date
rsm
67f0510edd rename ModelInference to Inference 2024-07-21 12:20:32 -07:00
Ashwin Bharambe
0df57c4447 fix bad merge with injection shield? 2024-07-20 23:54:44 -07:00
Ashwin Bharambe
ad62e2e1f3 make inference server load checkpoints for fp8 inference
- introduce quantization related args for inference config
- also kill GeneratorArgs
2024-07-20 22:54:48 -07:00
Ashwin Bharambe
7d2c0b14b8 Changes from the main repo 2024-07-20 22:52:29 -07:00
Ashwin Bharambe
95781ec85d Add toolchain from agentic system here 2024-07-19 12:30:35 -07:00