How to Choose Your LLM?

How to choose the right large language model (for translation)

This guide is for beginners who want to use LLMs for translation.

If it helps, please consider giving the repo a star. Thanks.

English | 中文-Simplified Chinese

This article gives beginners a first overview of using large language models (LLMs) for translation.
The main example is translating visual novels (VN), but the same ideas work for most other translation tasks.

Recommended tools and resources:

FAQ-Frequent Ask Questions & Fast guide

These are Questions related to

Basic Concept
Models & Performance
Local VS Online
Local LLMs Model Collection
Deployment & GPU Specification

which is a fast guide on how to read this article.
Pls read if you can't find certain chapter you're looking for.

What are LLMs good at?

Compared with traditional Neural Machine Translation (NMT — classic seq2seq / Transformer models trained only for translation),
LLMs are not better at everything.

But LLMs are clearly stronger in several important areas, especially:
Context handling, Controlling proper nouns / glossary, and Cross-lingual ability:

Context Handling
LLMs usually have long context windows. With suitable prompts, they can:
- Keep the story or conversation coherent
- Resolve pronouns and omitted parts (“he / she / that person / there”)
- Maintain consistent terminology and style over long texts
Controlling proper nouns/glossary/GPT-DICTIONARY (Proper Noun Translation)
LLMs are good at following user instructions, such as:
- Terminology rules
- Tone and style
- Pronouns (he / she / they)
- Custom dictionary explanations

For VN translation (or any task where context matters), LLMs often feel better overall than traditional NMT.

They can understand your prompts/instructions (depending on the model) and adjust translation accordingly.

Traditional NMT usually cannot handle pronouns correctly even with a dictionary, because it does not truly “understand” context.

LLMs can read explanations like:
James: A man's name

and treat that as context, so later they are more likely to use “he” instead of “she”.

You can also control style via prompt (formal, casual, cute, etc.), depending on the model’s ability.

Cross-lingual ability
Because modern LLMs are pretrained on large multilingual datasets, newer models usually:
- Support more languages easier
- Transfer knowledge across languages
- Work across different domains (everyday language, games, documents, etc.)

How to choose an LLM? Online vs Local

What you choose depends on your needs and trade-offs across the following factors. Here’s a quick guide:

Only Want to use local models but unsure about the Vram:
LLM VRAM USAGE LISTS

LLMs can be:

Local models
Online models
- Official APIs (online)
- Third-party hosted APIs (online)

Speed / Price / Performance
"CUDA out of money, pls charge💸"

Local (self-hosted) means you run the LLM on your own machine.
How large a model you can run mainly depends on:

GPU memory bandwidth
VRAM size
Compute (CUDA cores, etc.)

Whether you use local or online API, there are cost considerations:

Local: you "pay" for hardware (GPU, electricity, etc.).
Online: you "pay per token" to API providers.
(Tokens ≈ pieces of text; more text → more tokens.)

In general:

Bigger / newer models → better quality and/or speed → need more VRAM locally

Google offers a free-tier API. For example, Gemini 2.0 / 2.5 Flash often performs better than most small open models, but:

Free-tier has quota limits (requests per minute/day, tokens per minute/day).
- Larger models usually have tighter limits.
newer models have more strict censorship.(explained in the next section)

If you do NOT have an 8GB+ GPU,

(6GB is often the bare minimum; on Mac no less than 16GB unified memory is recommended)
- M series, not intel CPU

you can consider platforms like:

Free LLM API Provider

Some CN platforms also provide limited free tokens （check the article above).

With a 6GB+ VRAM GPU, you may try local deployment. See:

Usually:

Models smaller than ~4B parameters struggle with high translation quality.
Around 8B is a good “sweet spot” between quality and resource usage.

Privacy / Moderation

Local models
- Data stays on your machine (no external upload).
- Only moderation/censorship comes from the model itself; you can choose fine-tuned models with weaker or almost no moderation.
Google AI Studio / free-tier API:
- privacy isn’t a concern because there’s no private at all (lol).
  - Google states free-tier data may be used for training; there’s no opt-out.
Paid APIs:
True end-to-end encryption (E2EE) is not yet standard for LLM inference. TEE/Confidential Computing (CC-ON) depends on the cloud provider.
- Most providers offer:
  - Encryption in transit (HTTPS)
  - Encryption at rest
    - But plaintext is visible in memory on the server while the model runs.
- Providers usually promise to:
  - Protect data
  - Delete it regularly
  - Offer opt-out from training
Third-party hosted open-source models:
- You can choose fine-tuned variants that reduce or disable moderation.
- Platform-level logging and moderation depend on the host.

Multilingual / Cross-lingual ability

Multilingual strength depends on:
- The base model’s training data (which languages and domains)
- Model size
- Training techniques and fine-tuning
Domain-/language-tuned models strengthen some areas compared with their base model.
- Example: SakuraLLM
  - Continues pretraining and fine-tuning on general Japanese and light-novel / Galgame bilingual (ZH–JP) data.
  - Performance improves sharply in these domains (e.g. VN translation, EN↔JP).
    - But performance in other domains/languages is dramatically degraded, loses most of the base model (Qwen2.5) other capabilities.
      Which only can be used for translation tasks in those specific areas.

Thinking models

Thinking mode aims to strengthen:
- Contextual understanding
- Instruction following

In theory, this means better whole-document translation when long-range context is important (compared with non-thinking mode).

Actual quality still largely depends on data and training.

However, thinking mode （often chain-of-thought) requires extra tokens,
This increases token usage and latency, so it is usually not ideal for real-time translation.

Online models (Closed-source models)

Price & Free API
Different models have different prices

Cost = input tokens * unit price + output tokens (+thinking tokens) * unit price + (cache*unit price)
- 100 tokens ≈ 75 English words

Google provides a free-tier API (check details in the link).
But it’s limited by:

Requests per minute (RPM)

Tokens per minute (TPM)

Requests per day (RPD)
Daily quota resets at midnight Pacific Time.

Check the above link for rate limit details.

* AI Studio vs API: AI Studio usage is “free forever,” but its UI quotas don’t always match the API docs exactly; Google adjusts them periodically.

Performance
New closed-source models outperform open models in many areas

Check Timeline here

Representatives (as of 2026/05):

xAI (Grok-4.2/4.3)
OpenAI (GPT-5.5 Thinking)
Google (Gemini 3/3.1 Pro/3.5 Flash)
Anthropic (Claude 4.6/4.7)
Within the same generation of family, larger models are usually stronger (Gemini 3 Pro > Flash > Lite).

See various benchmarks on Kaggle.

Moderation
Closed-source models generally implement stricter safety moderation (which is reasonable)

Rough moderation strictness (weaker → stronger):
Grok-3/4 (easier to bypass) ≤ Claude 3.7 ≤ Gemini 2.0 series < Gemini 2.5 series < Gemini 3 series <<< OpenAI (newer the model, harder the bypass)

Gemini free tier appears stricter.
Web/client Chat versions are stricter than APIs.
Third-party hosting tends to be less strict than official APIs.

Multilingual / Cross-lingual ability
Leading closed-source models are typically strong across languages

Model	Verifiable count	Basis (scope)	Notes
xAI Grok-3~4.3	27 -	Official language list (model) -	Vendor-listed supported text languages. Undisclosed.
OpenAI GPT-4 GPT-4o	26 More than GPT-4	Evaluation coverage (translated MMLU) Official statement	Eval includes 26 languages; eval ≠ upper bound.
OpenAI GPT-5~5.5	60	*ChatGPT currently supports the following languages: 60	59 + English
Google Gemini 2.5 Pro Gemini 3+	37 / 40+ 109	Dev prompt languages / Web app UI Offical statement	Two counts: dev list 37; Web UI “40+”.
Anthropic Claude 3.7 / 4 / 4.1 Claude Sonnet 4.6	15 / 11 42	Official eval coverage / Product/UI languages Claude Sonnet 4.6 System Card	Eval: English + 14 non-English (15 total); UI languages: 11 GMMLU

Grok-3

English, Spanish, French, Afrikaans, Arabic, Bengali, Welsh, German, Greek, Indonesian, Icelandic, Italian, Japanese, Korean, Latvian, Marathi, Nepali, Punjabi, Polish, Russian, Swahili, Telugu, Thai, Turkish, Ukrainian, Urdu, Chinese

GPT-5~5.5

English, Albanian, Amharic, Arabic, Armenian, Bengali, Bosnian, Bulgarian, Burmese, Catalan, Chinese, Croatian, Czech, Danish, Dutch, Estonian, Finnish, French, Georgian, German, Greek, Gujarati, Hindi, Hungarian, Icelandic, Indonesian, Italian, Japanese, Kannada, Kazakh, Korean, Latvian, Lithuanian, Macedonian, Malay, Malayalam, Marathi, Mongolian, Norwegian, Persian, Polish, Portuguese, Punjabi, Romanian, Russian, Serbian, Slovak, Slovenian, Somali, Spanish, Swahili, Swedish, Tagalog, Tamil, Telugu, Thai, Turkish, Ukrainian, Urdu, Vietnamese

Gemini 3+

Afrikaans, Albanian, Amharic, Arabic, Armenian, Assamese, Azerbaijani, Basque, Belarusian, Bengali, Bosnian, Bulgarian, Catalan, Cebuano, Chinese (Simplified and Traditional), Corsican, Croatian, Czech, Danish, Dhivehi, Dutch, English, Esperanto, Estonian, Filipino (Tagalog), Finnish, French, Frisian, Galician, Georgian, German, Greek, Gujarati, Haitian Creole, Hausa, Hawaiian, Hebrew, Hindi, Hmong, Hungarian, Icelandic, Igbo, Indonesian, Irish, Italian, Japanese, Javanese, Kannada, Kazakh, Khmer, Korean, Krio, Kurdish, Kyrgyz, Lao, Latin, Latvian, Lithuanian, Luxembourgish, Macedonian, Malagasy, Malay, Malayalam, Maltese, Maori, Marathi, Meiteilon (Manipuri), Mongolian, Myanmar (Burmese), Nepali, Norwegian, Nyanja (Chichewa), Odia (Oriya), Pashto, Persian, Polish, Portuguese, Punjabi, Romanian, Russian, Samoan, Scots Gaelic, Serbian, Sesotho, Shona, Sindhi, Sinhala (Sinhalese), Slovak, Slovenian, Somali, Spanish, Sundanese, Swahili, Swedish, Tajik, Tamil, Telugu, Thai, Turkish, Ukrainian, Urdu, Uyghur, Uzbek, Vietnamese, Welsh, Xhosa, Yiddish, Yoruba, Zulu

Claude

English (baseline), Spanish, Portuguese (Brazil), Italian, French, Indonesian, German, Arabic, Chinese (Simplified), Korean, Japanese, Hindi, Bengali, Swahili, Yoruba

Local models (Open-weight / Open-source)

What open-source & open-weight means

Today, “open-source models” usually means open-weight.
Open-weight: the model weights are released; anyone can use/modify/fine-tune/commercialize them under a license (e.g., MIT).
Truly open-source: training/inference code & pipeline, model architecture, weights, and data sources are published, sufficient for full reproduction.

Where to find models?
Hugging Face
- CN platforms include ModelScope (魔搭社区).
How to read model names?
Example: Qwen3-8B-Thinking-2507-abliterated-Q8_0-gguf
(This model name is hypothetical for illustration.)
- Qwen3: base model family
- 8B: parameter count. Common sizes: 4B±; 8B±; 14B±; 30B±; 70B±; 100B+
  - MoE models (e.g., 30B-A3B): 30B is total params; A3B are the active params.
  - For VRAM vs params, see LLM VRAM USAGE LISTS
- Thinking: indicates a thinking-mode model (not all such models are labeled).
- 2507: time tag (2025/07), often small updates.
- abliterated: a style of “de-safety-filtered” fine-tune; typically means moderation is reduced or removed.
  - You’ll also see
    - uncensored/NSFW/amoral/evil/heretic(heresy)/, etc
- Q8_0 (gguf): a llama.cpp quantization type. See the link:
  - LLM VRAM USAGE LISTS
    - GGUF Quantization types & relative quality

Choosing an appropriate parameter size

Typically, GPU VRAM > model file size + context memory usage.
- Context can be stored in ram with slower token speed.
Larger model(parameters) × more data × more compute → better results (scaling law).
- For example: Qwen3-32B > Qwen3-14B > Qwen3-8B > Qwen3-4B
Common open-model size tiers: 4B±; 8B±; 14B±; 32B±; 70B±; 100B++
- <4B models often means poor translation quality — you can still test on Hugging Face or LMArena if curious.

Note: A conservative VRAM estimate is “model file size + ~1GB headroom.” Longer context or placing KV cache on-GPU requires more VRAM.

If you exceed VRAM (spilling into system RAM), speed drops sharply (except on unified-memory systems).

In theory, larger parameters models translate better; family-to-family comparisons are tricky.

Quantization & VRAM guide (examples)

Using Qwen3 GGUF sizes with llama.cpp as examples.
Here we only discuss GGUF models supported by llama.cpp (deployable via Ollama / LM Studio etc.). (MLX is similar.)

Qwen3-8B (GGUF)

Quantization	Model file size	Recommended VRAM (weights + headroom)
Q4_K_M	5.03 GB	≥ 6 GB
Q5_0	5.72 GB	≥ 7/8 GB
Q5_K_M	5.85 GB	≥ 7/8 GB
Q6_K	6.73 GB	≥ 8 GB
Q8_0	8.71 GB	≥ 10 GB

Qwen3-14B (GGUF)

Quantization	Model file size	Recommended VRAM (weights + headroom)
Q4_K_M	9.00 GB	≥ 11/12 GB
Q5_0	10.3 GB	≥ 12 GB
Q5_K_M	10.5 GB	≥ 12 GB
Q6_K	12.1 GB	≥ 14/16 GB
Q8_0	15.7 GB	≥ 18/20 GB

See more examples for different sizes here | LLM VRAM USAGE LISTS
* On Apple Mac (M1+), unified memory = RAM + VRAM. After subtracting 6–8GB for system/apps, the remainder approximates usable VRAM (many optimizations may need more memory).

For Apple (M series) by default you can only use 75% of total unified memory as vram.

* For MoE (e.g., Qwen3-30B-A3B): total params = 30B, all must load; A3B are just the active params.

GGUF quantization types & relative quality

Link

Note: “Quality” here is an experience-based tiering of overall closeness to FP16 + common community metrics (e.g., perplexity/objective evals). The same quant level can vary by model/task. _K denotes newer K-quant; _S/_M are mixed strategies—_M is usually higher quality than _S.

Quantization	Theoretical bpw*	Relative quality (vs FP16)	Typical use / advice	Notes
Q8_0	≈ 8.0	Highest (near FP16)	When VRAM is plenty and you want FP16-like fidelity or rigorous evals	“Legacy” method but the highest-quality quant; negligible drop vs FP16
Q6_K	6.5625	Very high (near Q8_0)	Aim for high quality while saving VRAM	K-Quant; better quality/size than same-bit legacy
Q5_K_M	5.5	High	A great 5-bit default; deployment “sweet spot”	`_M` prioritizes quality over `_S`
Q5_0	≈ 5.0	Medium–High	Consider only for old workflows	Legacy; usually worse than Q5_K_*
Q4_K_M	4.5	Medium (best among many 4-bit options)	Tight VRAM yet usable quality; common balance	Often the 4-bit go-to
Q4_K_S	≈ 4.5	Medium–Low (lower than `_M`)	When you need speed/smaller size in 4-bit	More aggressive mix, slight quality drop
Q4_0	≈ 4.0	Medium–Low	Compatibility/contrast only	Legacy; generally not recommended*
Q3_K_M	3.4375	Low	Extreme memory/edge devices	Noticeable degradation; not recommended

Optimize local model speed

Quick notes here—see the link above for details

Using LM STUDIO as an example:

Context Length
Max tokens the model can “remember/process” per run (prompt + generated output). Larger windows handle longer texts but require much more RAM/VRAM and compute; exceeding the true max can cause odd outputs.
GPU Offload
How many layers run on GPU (llama.cpp n_gpu_layers / -ngl; -1 or >#layers ≈ full offload). More GPU layers boost throughput but are VRAM-limited; the rest runs on CPU.
CPU Thread Pool Size
Number of CPU threads during inference (n_threads). More threads ≠ linear gains; near physical cores/system default is typical.
Evaluation Batch Size (n_batch)
How many tokens are fed in parallel during prefill. Larger batch → higher throughput but more RAM/VRAM; semantics unaffected.
RoPE Frequency Base
Controls rotary position encoding base frequency; sometimes increased for longer-context stability (model/impl dependent).
RoPE Frequency Scale
Scaling factor for RoPE; used with the above for extending effective context.
Offload KV Cache to GPU Memory
Put K/V cache and KQV ops on GPU to ease CPU/RAM pressure and speed up long contexts; availability varies.
Keep Model in Memory
Avoid unloading weights between calls for snappier reuse; costs RAM.
Try mmap()
Map weights from disk on demand; speeds loading and reduces resident memory. (llama.cpp uses mmap by default; can disable if needed.)
Seed
Fix randomness for reproducibility.
Flash Attention (experimental/optional)
A more efficient attention implementation that reduces memory traffic and speeds long-sequence inference; availability varies.
K Cache Quantization Type (experimental)
Quantize the K cache to cut memory/VRAM with minimal loss; stability varies by backend/hardware.
V Cache Quantization Type (experimental)
Same idea for the V cache; often paired with K quantization for long contexts on small GPUs.
Speculative Decoding | LM STUDIO intro
Use a small, fast “draft” model to propose tokens that the large “main” model quickly verifies; improves speed without changing the final distribution.

In most cases, keeping RoPE/KV defaults is fine; some models are sensitive.

Translation tools?

Make fan-patches or translation files

LunaTranslator — an all-in-one translator for visual novels/galgames. Supports text hooking (HOOK/OCR/clipboard/ASR/file translation), multiple online/local engines, pre-translation & caching, Python extensions; plus TTS, Japanese tokenization & furigana, dictionaries (MDICT/online), Anki flashcards, Yomitan plugins, etc.
AiNiee — one-click AI long-text translator. Fits common game text workflows (MTool, Ren’Py, Translator++), and many formats (i18next, EPUB/TXT, SRT/VTT/LRC, Word/PDF/Markdown). Auto file & language detection, context consistency & glossary, AI polishing/layout/term extraction; configurable for online and local models.
LinguaGacha — “nearly zero-config” multi-language text translator. Supports subtitles/ebooks/game texts, compatible with many online/local models (Claude/ChatGPT/etc.), emphasizes speed and preservation of format/code style. Many WOLF/Ren’Py/RPGMaker/Kirikiri games can be “translate-and-play,” with CLI and guides.
BallonsTranslator — DL-assisted translation & typesetting for comics/webtoons. One-click MT, WYSIWYG text editing (find/replace, batch styles), image editing & inpainting (masks/heal), OCR detection; Windows packaged build or Python source; compatible with many MT/LLM and offline models.

Deploying Local Models

Quantized-model deployment example (llama.cpp).
Below are common, practical platforms:

Possible platforms

LM Studio
- Official Guide
- All-in-one: local desktop app + OpenAI-compatible local server; easy setup (“install and go”).
- Built-in model discovery/download (Hugging Face); LAN sharing.
- Includes RAG, MCP integrations, and multiple GPU runtimes (CUDA/Metal/Vulkan/ROCm).
Ollama
- Official Guide
- Ollama Github
  - API Reference
- “Docker-for-models” local/LAN runtime and CLI/REST API. Supports Modelfile customization and model management.
- In 2025, official GUI (Win/mac) released; offers unified cloud + local options.
  - Ollama cloud models
koboldcpp
- Focused on writing/RP/fan-fic workflows; ships KoboldAI Lite UI (memory, world facts, character cards, scenes, etc.), multiple modes (chat/adventure/instruct/storywriter).
- Beyond text, includes TTS/ASR and Stable Diffusion image generation; multiple compatible APIs (OpenAI/Ollama compatible).
- Provides OpenAI/Ollama/Kobold compatible endpoints.

Other Platform

Model size / quantization spec choices
See LLM VRAM USAGE LISTS

Which model should I pick?
See above:

“How to choose an LLM?”
“Open-weight/Open-source local models——Which models are better to use?”

Name		Name	Last commit message	Last commit date
Latest commit History 276 Commits
FAQ.md		FAQ.md
FAQ_EN.md		FAQ_EN.md
Freellmapi.md		Freellmapi.md
LM STUDIO.png		LM STUDIO.png
Local_LLM_Models_Recommodation.md		Local_LLM_Models_Recommodation.md
Model_Speed.md		Model_Speed.md
Model_Speed_EN.md		Model_Speed_EN.md
Nvidia_GPU_Specification.md		Nvidia_GPU_Specification.md
OtherModels_gguf.md		OtherModels_gguf.md
README.md		README.md
READMECN.md		READMECN.md
README_en.md		README_en.md
README_en_original.md		README_en_original.md
TIMELINE 1.jpg		TIMELINE 1.jpg
timeline 1 DLC.png		timeline 1 DLC.png

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

How to Choose Your LLM?

English | 中文-Simplified Chinese

FAQ-Frequent Ask Questions & Fast guide

Table of Contents

What are LLMs good at?

How to choose an LLM? Online vs Local

Online models (Closed-source models)

Local models (Open-weight / Open-source)

Qwen3-8B (GGUF)

Qwen3-14B (GGUF)

Optimize local model speed

Translation tools?

Deploying Local Models

AI Timeline

About

Releases

Packages

Contributors

Folders and files

Latest commit

History

Repository files navigation

How to Choose Your LLM?

English | 中文-Simplified Chinese

FAQ-Frequent Ask Questions & Fast guide

Table of Contents

What are LLMs good at?

How to choose an LLM? Online vs Local

Online models (Closed-source models)

Local models (Open-weight / Open-source)

Qwen3-8B (GGUF)

Qwen3-14B (GGUF)

Optimize local model speed

Translation tools?

Deploying Local Models

AI Timeline

About

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors