---
datasets:
- samirmsallem/argumentative_student_peer_reviews
language:
- de
metrics:
- accuracy
base_model:
- deepset/gbert-base
pipeline_tag: text-classification
library_name: transformers
model-index:
- name: checkpoints
  results:
  - task:
      name: Text Classification
      type: text-classification
    dataset:
      name: samirmsallem/argumentative_student_peer_reviews
      type: samirmsallem/argumentative_student_peer_reviews
    metrics:
    - name: Accuracy
      type: accuracy
      value: 0.7708649468892261
---

## Text classification model for claim/premise detection in essay feedback


**gbert-base-claim_premise** is a text classification model in the non-scientific domain in German, finetuned from the model [gbert-base](https://huggingface.co/deepset/gbert-base). 
It was trained using a [annotated dataset](https://huggingface.co/datasets/samirmsallem/argumentative_student_peer_reviews) containing claim and premise sentences from essay feedback. The dataset was created by T. Wambsganss, C. Niklaus, M. Söllner, S. Handschuh and J. M. Leimeister and is available [here](https://github.com/thiemowa/argumentative_student_peer_reviews/).


### Training

Training was conducted on a 10 epoch fine-tuning approach, however this repository contains the results of the second epoch, since it has the best accuracy:

| Epoch | Eval Loss  | Accuracy   | 
| ----- | ---------- | ---------- |
| 1.0   | 0.4946     | 0.7621     |      
| 2.0   | **0.5074** | **0.7709** | 
| 3.0   | 0.8148     | 0.7627     |        
| 4.0   | 1.1393     | 0.7560     |               
| 5.0   | 1.3645     | 0.7551     |                
| 6.0   | 1.5397     | 0.7560     |                
| 7.0   | 1.8195     | 0.7548     |                
| 8.0   | 2.0723     | 0.7536     |                
| 9.0   | 2.0844     | 0.7566     |                
| 10.0  | 2.1382     | 0.7563     |                

In relation to the dataset, the model demonstrates that it can effectively learn to distinguish between the two classes claim and premise. However, the rapid onset of overfitting after epoch 2 suggests that the dataset is imbalanced and noisy. Further work should enable the model to be trained on more robust data to ensure better evaluation results.

### Text Classification Tags

|Text Classification Tag| Text Classification Label | 
| :----:                |    :----:                 | 
| 0                     | CLAIM          |
| 1                     | PREMISE          |