| --- |
| license: apache-2.0 |
| datasets: |
| - agentica-org/DeepScaleR-Preview-Dataset |
| base_model: |
| - Qwen/Qwen3-4B |
| language: |
| - en |
| pipeline_tag: text-generation |
| library_name: transformers |
| tags: |
| - moe |
| - text-generation-inference |
| - code |
| - deepscale |
| - math |
| --- |
| |
|  |
|
|
| # Segue-Qwen3\_DeepScaleR-Preview |
| |
| > Segue-Qwen3\_DeepScaleR-Preview is an experimental fine-tuned variant of the Qwen3-4B model architecture. It is trained on the DeepScaleR-Preview dataset—comprising high-quality mathematical reasoning problems—to achieve exceptional performance in symbolic, mathematical, and logical tasks with lightweight computational requirements. |
|
|
| ## Key Features |
|
|
| 1. Precision Reasoning with DeepScaleR-Preview Dataset |
| Fine-tuned on approximately 40,000 curated math problem-answer pairs sourced from: |
|
|
| * AIME (1984–2023) |
| * AMC (pre-2023) |
| * Omni-MATH |
| This enables superior symbolic manipulation and step-by-step logical deduction. |
| |
| 2. Lightweight Code Understanding |
| Capable of interpreting and generating correct code in Python, C++, and other logic-intensive languages with an emphasis on problem-solving and structured thought. |
|
|
| 3. Structured Output Formatting |
| Outputs are designed to be well-formatted in Markdown, JSON, LaTeX, or tables—ideal for technical documentation, math notebooks, and data workflows. |
|
|
| 4. Instruction-Following Accuracy |
| Strong multi-step instruction adherence, particularly for STEM domains. Ensures continuity, factual correctness, and process transparency in reasoning chains. |
|
|
| 5. Multilingual Capabilities |
| Supports over 20 languages for mathematical and logical reasoning, technical instruction translation, and cross-lingual academic support. |
|
|
| 6. Efficient 4B Architecture |
| Built on the Qwen3-4B base model to balance performance and scalability. Runs efficiently on mid-range GPUs while delivering high-accuracy inference. |
|
|
| ## Quickstart with Transformers |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| model_name = "prithivMLmods/Segue-Qwen3_DeepScaleR-Preview" |
| |
| model = AutoModelForCausalLM.from_pretrained( |
| model_name, |
| torch_dtype="auto", |
| device_map="auto" |
| ) |
| tokenizer = AutoTokenizer.from_pretrained(model_name) |
| |
| prompt = "Solve for x: 5(x - 2) = 3x + 4, showing all steps clearly." |
| |
| messages = [ |
| {"role": "system", "content": "You are a precise mathematical assistant trained on DeepScaleR-Preview dataset."}, |
| {"role": "user", "content": prompt} |
| ] |
| |
| text = tokenizer.apply_chat_template( |
| messages, |
| tokenize=False, |
| add_generation_prompt=True |
| ) |
| |
| model_inputs = tokenizer([text], return_tensors="pt").to(model.device) |
| |
| generated_ids = model.generate( |
| **model_inputs, |
| max_new_tokens=512 |
| ) |
| generated_ids = [ |
| output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids) |
| ] |
| |
| response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0] |
| print(response) |
| ``` |
|
|
| ## Intended Use |
|
|
| * Step-by-step mathematical problem solving |
| * Symbolic computation and logic derivation |
| * Code generation and correction in technical environments |
| * Automated LaTeX/Markdown/JSON generation for education and documentation |
| * Academic tutoring and educational assistants |
| * Multilingual reasoning and translation of structured content |
|
|
| ## Limitations |
|
|
| * Less suitable for open-domain conversation or creative writing |
| * Smaller context window compared to large-scale LLMs |
| * May be sensitive to token formatting in edge-case symbolic prompts |
| * Could underperform on intentionally adversarial logic inputs |
|
|
| ## References |
|
|
| 1. Qwen2.5 Technical Report – [https://arxiv.org/pdf/2412.15115](https://arxiv.org/pdf/2412.15115) |
| 2. YaRN: Context Window Extension for LLMs – [https://arxiv.org/pdf/2309.00071](https://arxiv.org/pdf/2309.00071) |
