Julian-600M Instruct (SFT v2)

600M parameter language model fine-tuned on 2.47M instruction examples using ChatML format with response-only loss masking.

Base model trained from scratch on 39B tokens (70% EN, 30% FR) on Google Cloud TPUs with JAX.

Base Model | Model Card | Base Demo
50 500
0.1 1.5
0.1 1
1 2

Model Details

Parameter Value
Base Model Julian-600M-40B (53.5% HellaSwag)
SFT Data 2.47M examples (OpenHermes, UltraChat, OASST-FR)
SFT Steps 30,000 (response-only loss)
Format ChatML (<|im_start|>, <|im_end|>)
Architecture LLaMA-style (RoPE, SwiGLU, RMSNorm)
Context Length 2048 tokens

Note: This is a 600M parameter model. Responses may contain factual errors or inconsistencies. The model performs better in English than French due to training data distribution.

Trained with Google TPU Research Cloud (TRC) program