🇮🇳 Indra — Indian Language AI Assistant

Indra is a fine-tuned LLM built on Qwen2.5-Coder-1.5B-Instruct, trained on Indian language and history datasets. It can converse in 10 Indian languages, answer questions about Indian history & culture, and still write code.

✨ Highlights

• 🗣️ 10 Indian languages — Hindi, Bengali, Tamil, Telugu, Marathi, Gujarati, Kannada, Malayalam, Punjabi, Odia
• 📜 Indian history & culture — From Indus Valley to modern India
• 🔄 Bilingual conversations — Hinglish, code-switching, vernacular queries
• 💻 Coding preserved — Still writes Python, JavaScript, and full-stack code
• 📦 Lightweight — 1.5B parameters, runs on consumer GPUs

Quick Start

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("RockySinghRajput/Indra", torch_dtype="auto", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("RockySinghRajput/Indra")

messages = [
    {"role": "system", "content": "आप Indra हैं, एक बुद्धिमान AI सहायक।"},
    {"role": "user", "content": "भारत के स्वतंत्रता संग्राम के बारे में बताइए।"}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)

output = model.generate(inputs.input_ids, max_new_tokens=512, temperature=0.7, top_p=0.9)
print(tokenizer.decode(output[0][len(inputs.input_ids[0]):], skip_special_tokens=True))

Hinglish Example

messages = [
    {"role": "system", "content": "You are Indra, a helpful AI that understands Hindi and English."},
    {"role": "user", "content": "Mujhe Python mein ek calculator banana hai, kaise karoon?"}
]

Model Details

Property	Value
Base Model	Qwen/Qwen2.5-Coder-1.5B-Instruct
Parameters	1.5B
Type	Causal Language Model (merged LoRA fine-tune)
Languages	English, Hindi, Bengali, Tamil, Telugu, Marathi, Gujarati, Kannada, Malayalam, Punjabi, Odia
License	Apache 2.0
Developed by	RockySinghRajput
Related Model	IndraCoder (coding-focused version)

Supported Languages

Language	Script	Code
Hindi	देवनागरी	hi
Bengali	বাংলা	bn
Tamil	தமிழ்	ta
Telugu	తెలుగు	te
Marathi	मराठी	mr
Gujarati	ગુજરાતી	gu
Kannada	ಕನ್ನಡ	kn
Malayalam	മലയാളം	ml
Punjabi	ਪੰਜਾਬੀ	pa
Odia	ଓଡ଼ିଆ	or
English	Latin	en

Training Details

Training Data

Fine-tuned on 6 Indian language & culture datasets:

Dataset	Purpose	Content
CohereForAI/aya_dataset	Multilingual Indian conversations	10 Indian languages
Cognitive-Lab/Hindi-Instruct	Hindi instruction following	Hindi Q&A
sarvamai/samvaad-hi-en-instruct-v2	Bilingual conversations	Hindi-English
CohereForAI/aya_collection (India-filtered)	Indian history & culture	History, heritage, knowledge
CohereForAI/aya_collection (Hindi WikiQA)	Hindi knowledge base	Wikipedia-sourced Hindi QA
ai4bharat/IndicSentiment	Hindi language understanding	Sentiment analysis

Indian History Coverage

The model has been trained on Indian history spanning:

• Ancient India — Indus Valley Civilization, Vedic period, Maurya & Gupta Empires
• Medieval India — Delhi Sultanate, Mughal Empire, Vijayanagara, Maratha Empire, Bhakti & Sufi movements
• Modern India — British Raj, Freedom struggle, Independence, Republic
• Indian Constitution — Fundamental rights, governance, democracy
• Culture & Heritage — Art, architecture, literature, philosophy, classical music, Ayurveda, Yoga

Training Procedure

• Method: LoRA (Low-Rank Adaptation) → merged into base model
• LoRA Config: r=16, alpha=16, dropout=0.05
• Target Modules: q_proj, k_proj, v_proj, o_proj
• Epochs: 2
• Learning Rate: 2e-5 (lower to preserve base capabilities)
• Optimizer: paged_adamw_8bit
• Sequence Length: 512 tokens
• Precision: FP16 mixed precision
• Quantization: 4-bit NF4 (QLoRA) during training

Compute Infrastructure

• Hardware: NVIDIA T4 GPU
• Training Time: ~2 hours

Capabilities

✅ What Indra Can Do

• Converse in Indian languages — Answer questions in Hindi, Bengali, Tamil, Telugu, and more
• Indian history & culture — Detailed knowledge of Indian civilization
• Hinglish/bilingual — Handle mixed Hindi-English naturally
• General knowledge — Science, geography, current affairs with Indian context
• Coding — Write code in Python, JavaScript, and other languages
• Sentiment analysis — Understand sentiment in Hindi text

⚠️ Limitations

• 1.5B model — Smaller than commercial LLMs; may produce shorter or less nuanced responses
• Script limitations — Stronger in Hindi/Devanagari; other Indian scripts may have lower quality
• Not a translator — Optimized for conversation, not professional translation
• May hallucinate — Always verify historical facts and generated content
• English-centric base — Indian language abilities are fine-tuned on top of an English-dominant base

❌ Out-of-Scope Use

• Professional translation services
• Legal or medical advice
• Factual source of record for academic research
• Generating harmful or culturally insensitive content

Evaluation

Test	Language	Task	Result
Hindi Chat	Hindi	Gandhi's role in freedom struggle	✅ Detailed response
Indian History	English	Gupta Empire Golden Age	✅ Accurate overview
Hinglish	Mixed	"Python mein calculator banana hai"	✅ Code + Hindi explanation
Coding	English	Binary search implementation	✅ Working code
Indian Culture	Hindi	Classical music ragas	✅ Cultural knowledge

Model Family

Model	Focus	Repo
Indra (this model)	Indian languages + history	RockySinghRajput/Indra
IndraCoder	Coding + debugging	RockySinghRajput/Indracoder
IndraCoder-7B	Advanced coding + chat (coming soon)	RockySinghRajput/IndraCoder-7B

Citation

@misc{indra2025,
  title={Indra: An Indian Language AI Assistant},
  author={RockySinghRajput},
  year={2025},
  publisher={HuggingFace},
  url={https://huggingface.co/RockySinghRajput/Indra}
}

Contact

• HuggingFace: RockySinghRajput