rdev610/fine-tuned-distilbert-movie-sentiment

Model Card for Model ID

This model classifies movie reviews as either positive or negative.

Model Details

Model Description

This model classifies movie reviews as either positive or negative. It was trained on the cornell-movie-reviews dataset and was based upon the dilbert-base uncased model.

• Developed by: Rushika Devineni
• Model type: distilbert
• Language(s) (NLP): English
• License: MIT License (Inherited from the base model: distilbert-base-uncased)
• Finetuned from model [optional]: distilbert-base-uncased model

Uses

The main use is analyze movie reviews and classify them as negative or positive. Users can input raw text (a movie review or similar piece of text) and the model will output a predicted sentiment label (either 'positive' or 'negative') along with associated confidence probabilities for each class. Foreseeable direct users include researchers, developers, or hobbyists who need a pre-trained and fine-tuned tool for sentiment analysis on textual data, particularly within or related to the film domain.

Direct Use

This model is intended for direct use as a sentiment classifier for English-language text, primarily focusing on the domain of movie reviews.

Out-of-Scope Use

This model was fine-tuned specifically for binary sentiment classification (positive/negative) of movie reviews from the Rotten Tomatoes dataset. Using it in the following contexts is out of scope and may lead to unreliable or biased results:

• Sentiment analysis on text from significantly different domains: This includes technical documentation, medical records, legal texts, financial reports, conversational dialogue (like social media comments without review-like structure), or news articles, where the language, tone, and expression of sentiment can vary greatly from movie reviews.
• Fine-grained sentiment analysis: The model is trained for binary (positive/negative) classification. It is not designed to detect neutral sentiment, emotions (like anger, joy, sadness), or specific aspects of a review that a sentiment might apply to (e.g., sentiment about the acting vs. the plot).
• Use in high-stakes decision-making: The model's predictions should not be used as the sole basis for critical decisions about individuals (e.g., in hiring, lending, or risk assessment). Sentiment analysis can be complex and prone to error or bias.
• Analysis of non-English text: The model is fine-tuned on English data and is not intended for use on other languages.
• Generation of text: This model is a classification model and cannot generate text.

Bias, Risks, and Limitations

Despite efforts to create fair and robust models, this fine-tuned sentiment analysis model, like most NLP models, may exhibit biases and limitations:

• Dataset Bias: The model was fine-tuned exclusively on the Rotten Tomatoes dataset. This dataset may contain biases related to specific film genres, directors, actors, or demographic groups mentioned in reviews. The linguistic style and common phrases in movie reviews may also introduce domain-specific biases that don't generalize to other text types.
• Inherited Bias from Pre-training: The base distilbert-base-uncased model was pre-trained on a large corpus of internet text, which can contain societal biases (related to gender, race, religion, etc.). Although fine-tuning adapts the model to the new task, some of these pre-existing biases might persist or influence predictions on sensitive topics.
• Difficulty with Nuance and Figurative Language: The model may struggle to correctly interpret sarcasm, irony, subtle sentiment, double negatives, or complex sentence structures where the overall sentiment is not immediately obvious from individual words.
• Generalization Limitations: Performance may degrade significantly when applied to text that is structurally or stylistically different from the movie reviews it was trained on.
• Subjectivity of Sentiment: Sentiment itself can be subjective and context-dependent, leading to potential disagreements between the model's prediction and human interpretation, especially for ambiguous reviews.

Recommendations

Users of this model (both direct and those integrating it into other systems) should be aware of its limitations and potential biases and consider the following recommendations:

• Understand the Domain: Be mindful that the model is optimized for movie review sentiment. Validate its performance on your specific data if it differs from movie reviews before relying on its predictions.
• Avoid High-Stakes Applications: Do not use the model for critical decisions about individuals or in sensitive contexts where misclassification could cause harm. Human review of predictions is recommended for important applications.
• Be Cautious with Nuance: Do not assume the model can perfectly handle sarcasm, irony, or highly nuanced language.
• Test for Bias: If applying the model to data involving different demographic groups or potentially sensitive topics, consider evaluating its predictions specifically on subsets of your data to identify and mitigate potential biases.
• Monitor Performance: Continuously monitor the model's performance on your specific task and data over time, as text distributions and sentiment expressions can change.
• Provide Context: When using the model's output, inform end-users that the sentiment is predicted by an AI model and may not always be accurate or capture the full complexity of the text.

How to Get Started with the Model

Use the code below to get started with the model.

from transformers import AutoTokenizer, AutoModelForSequenceClassification import torch

Define the model ID on the Hugging Face Hub

REPLACE [Your HF Username] with your actual username if it's different from rdev610

model_id = "rdev610/fine-tuned-distilbert-movie-sentiment" # Your model's ID on the Hub

Load the model and tokenizer from the Hub

try: tokenizer = AutoTokenizer.from_pretrained(model_id) model = AutoModelForSequenceClassification.from_pretrained(model_id) print(f"Model and tokenizer loaded successfully from {model_id}")

# Example of how to use the model for inference:
text = "This movie was fantastic and I loved every minute of it!" # Replace with your desired input text

# Prepare the input for the model
inputs = tokenizer(text, return_tensors="pt")

# If using GPU, move model and inputs to GPU
if torch.cuda.is_available():
    model.to('cuda')
    inputs = {key: val.to('cuda') for key, val in inputs.items()}

# Perform inference
model.eval() # Set the model to evaluation mode
with torch.no_grad(): # Disable gradient calculation
    outputs = model(**inputs)

# Interpret the output
# Assuming a binary classification (0 for negative, 1 for positive)
predictions = torch.argmax(outputs.logits, dim=1)
label_map = {0: 'negative', 1: 'positive'} # Confirm this mapping is correct
sentiment = label_map[predictions.item()]

print(f"\nInput Text: {text}")
print(f"Predicted Sentiment: {sentiment}")

except Exception as e: print(f"Error loading or using the model: {e}")

Training Details

Training Data

The model was fine-tuned on the Rotten Tomatoes (rtr) dataset, which is available on the Hugging Face datasets library (https://huggingface.co/datasets/rotten_tomatoes). This dataset consists of movie review snippets labeled with binary sentiment (positive or negative).

Training Procedure

Training Hyperparameters

• Training regime: [More Information Needed]

Speeds, Sizes, Times [optional]

[More Information Needed]

Evaluation

Testing Data, Factors & Metrics

Testing Data

[More Information Needed]

Factors

[More Information Needed]

Metrics

[More Information Needed]

Results

[More Information Needed]

Summary

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: [More Information Needed]
• Hours used: [More Information Needed]
• Cloud Provider: [More Information Needed]
• Compute Region: [More Information Needed]
• Carbon Emitted: [More Information Needed]

Technical Specifications [optional]

Model Architecture and Objective

[More Information Needed]

Compute Infrastructure

[More Information Needed]

Hardware

[More Information Needed]

Software

[More Information Needed]