phoenix-oss/llama-stack
1
0
Fork 0
forked from phoenix-oss/llama-stack-mirror
Code Pull requests Releases Packages 2 Activity Actions
llama-stack/llama_stack/providers/impls/meta_reference/inference
History
Ashwin Bharambe 09b793c4d6 Fix fp8 implementation which had bit-rotten a bit 
I only tested with "on-the-fly" bf16 -> fp8 conversion, not the "load
from fp8" codepath.

YAML I tested with:

```
providers:
  - provider_id: quantized
    provider_type: meta-reference-quantized
    config:
      model: Llama3.1-8B-Instruct
      quantization:
        type: fp8
```
2024-10-15 13:57:01 -07:00
..
quantization Fix fp8 implementation which had bit-rotten a bit 2024-10-15 13:57:01 -07:00
__init__.py Split off meta-reference-quantized provider 2024-10-10 16:03:19 -07:00
config.py Split off meta-reference-quantized provider 2024-10-10 16:03:19 -07:00
generation.py Fix fp8 implementation which had bit-rotten a bit 2024-10-15 13:57:01 -07:00
inference.py Split off meta-reference-quantized provider 2024-10-10 16:03:19 -07:00
model_parallel.py Split off meta-reference-quantized provider 2024-10-10 16:03:19 -07:00
parallel_utils.py Split off meta-reference-quantized provider 2024-10-10 16:03:19 -07:00