Editing Models with Task Arithmetic
Paper • 2212.04089 • Published • 8
How to use chargoddard/test-t5-merge with Transformers:
# Load model directly
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
tokenizer = AutoTokenizer.from_pretrained("chargoddard/test-t5-merge")
model = AutoModelForSeq2SeqLM.from_pretrained("chargoddard/test-t5-merge")This is a merge of pre-trained language models created using mergekit.
This model was merged using the Task Arithmetic merge method using google/flan-t5-large as a base.
The following models were included in the merge:
The following YAML configuration was used to produce this model:
merge_method: task_arithmetic
base_model: google/flan-t5-large
models:
- model: google/flan-t5-large
- model: Varshitha/flan-t5-large-finetune-medicine-v5
parameters:
weight: 0.75
- model: andgonzalez/flan-t5-large-samsum
parameters:
weight: 0.6
- model: google/flan-t5-large+jbochi/flan-t5-large-spelling-peft
parameters:
weight: 0.3