1.1 KiB
1.1 KiB
title, date, draft, tags, categories, summary, author
| title | date | draft | tags | categories | summary | author | ||||
|---|---|---|---|---|---|---|---|---|---|---|
| [myai] feat(myai): switch to Qwen2.5-Coder-1.5B (local GPU, ~30s/64tok) | 2026-08-14T22:02:59+02:00 | false |
|
|
Commit 533ae6e par Betty dans myai | Betty |
Commit 533ae6e
| Repository | myai |
| Branch | main |
| Author | Betty |
| Hash | 533ae6e0010dacfbd8c941d98dca96a0c84f8575 |
Description
Replace StarCoder2-3B with Qwen/Qwen2.5-Coder-1.5B:
- 1.5B params → ~3GB float16 → fits entirely in T400 4GB VRAM
- No CPU offloading → ~30s/64tok vs 3-7min before
- safetensors format → compatible with PyTorch 2.2 + transformers 4.57
- Not gated → no HF token needed for download
Removed hf_token secret dependency, restored GPU placement constraint, adjusted resource limits to 2CPU/3G RAM.
Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
Files Changed
M app/codegen.py
M stack/myai.yml
Diff Summary
app/codegen.py | 8 ++++----
stack/myai.yml | 6 +++---
2 files changed, 7 insertions(+), 7 deletions(-)