---
license: apache-2.0
base_model:
- OccultAI/Morpheus-8B-v2
- SicariusSicariiStuff/Assistant_Pepe_8B
tags:
- merge
- llama
- occult
- uncensored
- moe
datasets:
- OccultAI/Matrix_77
- SicariusSicariiStuff/UBW_Tapestries
language:
- en
library_name: transformers
widget:
  - text: "Meme-Trix MoE 14B A8B v2"
    output:
      url: https://cdn-uploads.huggingface.co/production/uploads/68e840caa318194c44ec2a04/hEyahuR43IMF5TC4XrHGS.png
---

> [!CAUTION]
> <span style="color:red; font-weight:bold">⚠️ Warning:</span> This model can produce narratives and RP that contain violent and graphic erotic content. Adjust your system prompt accordingly, and use **Llama 3** chat template.
>

# Meme-Trix MoE 14B A8B v2

![Meme-Trix](https://cdn-uploads.huggingface.co/production/uploads/68e840caa318194c44ec2a04/hEyahuR43IMF5TC4XrHGS.png)

A custom built Llama 3.1 8B MoE (Mixture of Experts) merge which combines Morpheus v2 with Assistant Pepe. This should be superior to v1 and produce more intelligent responses.

If you want to merge custom Llama MoE you can add these scripts to your mergekit environment:
- [mergekit-main\mergekit\architecture\moe_defs.py](https://huggingface.co/Naphula/Meme-Trix-MoE-14B-A8B-v1/blob/main/moe_defs.py)
- [mergekit-main\mergekit\moe\__init__.py](https://huggingface.co/Naphula/Meme-Trix-MoE-14B-A8B-v1/blob/main/__init__.py)
- [mergekit-main\mergekit\moe\llama.py](https://huggingface.co/Naphula/Meme-Trix-MoE-14B-A8B-v1/blob/main/llama.py)

Then assign the `num_experts_per_tok` in `config.json` (or the `config.yaml`)

## Recommended Settings
**(bolded kobold non-defaults)**
- **Temp 1.0**
- **TopNSigma 1.25**
- **Min-P 0.1**
- **Repetition Penalty 1.08**
- **Top-P 1.0**
- Top-K 100
- Top-A 0
- Typical Sampling 1
- Tail-Free Sampling 1
- Presence Penalty 0
- Sampler Seed -1
- Rp.Range 360
- Rp.Slope 0.7
- Smoothing Factor 0
- Smoothing Curve 1
- DynaTemp 0
- Mirostat Mode OFF ("2" enhances creativity but also errors)
- Mirostat Tau 5
- Mirostat Eta 0.1
- **DRY Multiplier 0.8**
- DRY Base 1.75
- DRY A.Len 2
- DRY L.Len 320
- **XTC Threshold 0.1**
- **XTC Probability 0.08** (The "Anti-Cliche" Shield)
- **DynaTemp ON** (The "Poor Man's Fading Mirostat")
- **Minimum Temperature 0.65**
- **Maximum Temperature 1.35**
- **Temperature 1.0**
- **DynaTemp-Range 0.35**
- **DynaTemp-Exponent 1**