Nikeytas commited on
Commit
eb8799c
·
verified ·
1 Parent(s): 8dc899e

Update model card with comprehensive documentation

Browse files
Files changed (1) hide show
  1. README.md +307 -189
README.md CHANGED
@@ -1,199 +1,317 @@
1
  ---
2
- library_name: transformers
3
- tags: []
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4
  ---
5
 
6
- # Model Card for Model ID
 
 
 
 
 
 
 
 
7
 
8
- <!-- Provide a quick summary of what the model is/does. -->
 
 
 
 
9
 
 
10
 
 
 
 
11
 
12
- ## Model Details
 
 
13
 
14
- ### Model Description
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
15
 
16
- <!-- Provide a longer summary of what this model is. -->
17
-
18
- This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
19
-
20
- - **Developed by:** [More Information Needed]
21
- - **Funded by [optional]:** [More Information Needed]
22
- - **Shared by [optional]:** [More Information Needed]
23
- - **Model type:** [More Information Needed]
24
- - **Language(s) (NLP):** [More Information Needed]
25
- - **License:** [More Information Needed]
26
- - **Finetuned from model [optional]:** [More Information Needed]
27
-
28
- ### Model Sources [optional]
29
-
30
- <!-- Provide the basic links for the model. -->
31
-
32
- - **Repository:** [More Information Needed]
33
- - **Paper [optional]:** [More Information Needed]
34
- - **Demo [optional]:** [More Information Needed]
35
-
36
- ## Uses
37
-
38
- <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
39
-
40
- ### Direct Use
41
-
42
- <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
43
-
44
- [More Information Needed]
45
-
46
- ### Downstream Use [optional]
47
-
48
- <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
49
-
50
- [More Information Needed]
51
-
52
- ### Out-of-Scope Use
53
-
54
- <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
55
-
56
- [More Information Needed]
57
-
58
- ## Bias, Risks, and Limitations
59
-
60
- <!-- This section is meant to convey both technical and sociotechnical limitations. -->
61
-
62
- [More Information Needed]
63
-
64
- ### Recommendations
65
-
66
- <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
67
-
68
- Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
69
-
70
- ## How to Get Started with the Model
71
-
72
- Use the code below to get started with the model.
73
-
74
- [More Information Needed]
75
-
76
- ## Training Details
77
-
78
- ### Training Data
79
-
80
- <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
81
-
82
- [More Information Needed]
83
-
84
- ### Training Procedure
85
-
86
- <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
87
-
88
- #### Preprocessing [optional]
89
-
90
- [More Information Needed]
91
-
92
-
93
- #### Training Hyperparameters
94
-
95
- - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
96
-
97
- #### Speeds, Sizes, Times [optional]
98
-
99
- <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
100
-
101
- [More Information Needed]
102
-
103
- ## Evaluation
104
-
105
- <!-- This section describes the evaluation protocols and provides the results. -->
106
-
107
- ### Testing Data, Factors & Metrics
108
-
109
- #### Testing Data
110
-
111
- <!-- This should link to a Dataset Card if possible. -->
112
-
113
- [More Information Needed]
114
-
115
- #### Factors
116
-
117
- <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
118
-
119
- [More Information Needed]
120
-
121
- #### Metrics
122
-
123
- <!-- These are the evaluation metrics being used, ideally with a description of why. -->
124
-
125
- [More Information Needed]
126
-
127
- ### Results
128
-
129
- [More Information Needed]
130
-
131
- #### Summary
132
-
133
-
134
-
135
- ## Model Examination [optional]
136
-
137
- <!-- Relevant interpretability work for the model goes here -->
138
-
139
- [More Information Needed]
140
-
141
- ## Environmental Impact
142
-
143
- <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
144
-
145
- Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
146
-
147
- - **Hardware Type:** [More Information Needed]
148
- - **Hours used:** [More Information Needed]
149
- - **Cloud Provider:** [More Information Needed]
150
- - **Compute Region:** [More Information Needed]
151
- - **Carbon Emitted:** [More Information Needed]
152
-
153
- ## Technical Specifications [optional]
154
-
155
- ### Model Architecture and Objective
156
-
157
- [More Information Needed]
158
-
159
- ### Compute Infrastructure
160
-
161
- [More Information Needed]
162
-
163
- #### Hardware
164
-
165
- [More Information Needed]
166
-
167
- #### Software
168
-
169
- [More Information Needed]
170
-
171
- ## Citation [optional]
172
-
173
- <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
174
-
175
- **BibTeX:**
176
-
177
- [More Information Needed]
178
-
179
- **APA:**
180
-
181
- [More Information Needed]
182
-
183
- ## Glossary [optional]
184
-
185
- <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
186
-
187
- [More Information Needed]
188
-
189
- ## More Information [optional]
190
-
191
- [More Information Needed]
192
-
193
- ## Model Card Authors [optional]
194
-
195
- [More Information Needed]
196
-
197
- ## Model Card Contact
198
 
199
- [More Information Needed]
 
 
 
1
  ---
2
+ license: mit
3
+ base_model: MCG-NJU/videomae-base
4
+ tags:
5
+ - video-classification
6
+ - crime-detection
7
+ - violence-detection
8
+ - videomae
9
+ - computer-vision
10
+ - security
11
+ - surveillance
12
+ - generated_from_trainer
13
+ language:
14
+ - en
15
+ datasets:
16
+ - jinmang2/ucf_crime
17
+ metrics:
18
+ - accuracy
19
+ - precision
20
+ - recall
21
+ - f1
22
+ pipeline_tag: video-classification
23
+ model-index:
24
+ - name: videomae-crime-detector-ultra-v1
25
+ results:
26
+ - task:
27
+ name: Violence Detection
28
+ type: video-classification
29
+ dataset:
30
+ name: UCF Crime Dataset (Subset)
31
+ type: jinmang2/ucf_crime
32
+ args: violence_detection
33
+ metrics:
34
+ - name: Accuracy
35
+ type: accuracy
36
+ value: 0.7188
37
+ - name: Precision
38
+ type: precision
39
+ value: 0.7207
40
+ - name: Recall
41
+ type: recall
42
+ value: 0.7188
43
+ - name: F1
44
+ type: f1
45
+ value: 0.7190
46
  ---
47
 
48
+ # Nikeytas/Videomae Crime Detector Ultra V1
49
+
50
+ This model is a fine-tuned version of [MCG-NJU/videomae-base](https://huggingface.co/MCG-NJU/videomae-base) on the UCF Crime dataset with **event-based binary classification**. It achieves the following results on the evaluation set:
51
+
52
+ - **Loss**: 1.4159
53
+ - **Accuracy**: 0.7188
54
+ - **Precision**: 0.7207
55
+ - **Recall**: 0.7188
56
+ - **F1 Score**: 0.7190
57
 
58
+ ## 🎯 Model Overview
59
+
60
+ This VideoMAE model has been fine-tuned for **binary violence detection** in video content. The model classifies videos into two categories:
61
+ - **Violent Crime** (1): Videos containing violent criminal activities
62
+ - **Non-Violent Incident** (0): Videos with non-violent or normal activities
63
 
64
+ The model is based on the **VideoMAE architecture** and has been specifically trained on a curated subset of the UCF Crime dataset with event-based categorization for realistic crime detection scenarios.
65
 
66
+ ## 📊 Dataset & Training
67
+
68
+ ### Dataset Composition
69
 
70
+ **Total Videos**: 600
71
+ - **Violent Crime Videos**: 300
72
+ - **Non-Violent Incident Videos**: 300
73
 
74
+ **Class Balance**: 50.0% violent crimes
75
+
76
+ **Event Distribution**:
77
+ - **Abuse**: 28 videos
78
+ - **Arrest**: 18 videos
79
+ - **Arson**: 16 videos
80
+ - **Assault**: 62 videos
81
+ - **Burglary**: 120 videos
82
+ - **Explosion**: 54 videos
83
+ - **Fighting**: 48 videos
84
+ - **RoadAccidents**: 58 videos
85
+ - **Robbery**: 184 videos
86
+ - **Shoplifting**: 36 videos
87
+ - **Stealing**: 46 videos
88
+ - **Vandalism**: 72 videos
89
+
90
+ **Data Splits**:
91
+ - **Training**: 384 videos
92
+ - **Validation**: 96 videos
93
+ - **Test**: 120 videos
94
+
95
+ ## 🎯 Performance
96
+
97
+ ### Performance Metrics
98
+
99
+ **Validation Performance**:
100
+ - **eval_loss**: 1.4159
101
+ - **eval_accuracy**: 0.7188
102
+ - **eval_precision**: 0.7207
103
+ - **eval_recall**: 0.7188
104
+ - **eval_f1**: 0.7190
105
+ - **eval_runtime**: 11.1870
106
+ - **eval_samples_per_second**: 8.5810
107
+ - **eval_steps_per_second**: 4.2910
108
+ - **epoch**: 15.0000
109
+
110
+ **Test Performance**:
111
+ - **eval_loss**: 1.7586
112
+ - **eval_accuracy**: 0.6833
113
+ - **eval_precision**: 0.6963
114
+ - **eval_recall**: 0.6833
115
+ - **eval_f1**: 0.6802
116
+ - **eval_runtime**: 13.9918
117
+ - **eval_samples_per_second**: 8.5760
118
+ - **eval_steps_per_second**: 4.2880
119
+ - **epoch**: 15.0000
120
+
121
+ **Training Information**:
122
+ - **Training Time**: 69.5 minutes
123
+ - **Best Accuracy Achieved**: 0.7188
124
+ - **Model Architecture**: VideoMAE Base (fine-tuned)
125
+ - **Fine-tuning Approach**: Event-based binary classification
126
+
127
+ ## 🚀 Training Procedure
128
+
129
+ ### Training Hyperparameters
130
+
131
+ The following hyperparameters were used during training:
132
+ - **Learning Rate**: 5e-05
133
+ - **Train Batch Size**: 2
134
+ - **Eval Batch Size**: 2
135
+ - **Optimizer**: AdamW with betas=(0.9,0.999) and epsilon=1e-08
136
+ - **LR Scheduler Type**: Linear
137
+ - **Training Epochs**: 15
138
+ - **Weight Decay**: 0.01
139
+
140
+ ### Training Results
141
+
142
+ | Training Loss | Epoch | Step | Validation Loss | Accuracy |
143
+ |---------------|-------|------|-----------------|----------|
144
+ | 0.71875 | 15.00 | N/A | 1.4159 | 0.7188 |
145
+
146
+ ### Framework Versions
147
+
148
+ - **Transformers**: 4.30.2+
149
+ - **PyTorch**: 2.0.1+
150
+ - **Datasets**: Latest
151
+ - **Device**: Apple Silicon MPS / CUDA / CPU (Auto-detected)
152
+
153
+ ## 🚀 Quick Start
154
+
155
+ ### Installation
156
+
157
+ ```bash
158
+ pip install transformers torch torchvision opencv-python pillow
159
+ ```
160
+
161
+ ### Basic Usage
162
+
163
+ ```python
164
+ import torch
165
+ from transformers import AutoModelForVideoClassification, AutoProcessor
166
+ import cv2
167
+ import numpy as np
168
+
169
+ # Load model and processor
170
+ model = AutoModelForVideoClassification.from_pretrained("Nikeytas/videomae-crime-detector-ultra-v1")
171
+ processor = AutoProcessor.from_pretrained("Nikeytas/videomae-crime-detector-ultra-v1")
172
+
173
+ # Process video
174
+ def classify_video(video_path, num_frames=16):
175
+ # Extract frames
176
+ cap = cv2.VideoCapture(video_path)
177
+ frames = []
178
+
179
+ total_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
180
+ indices = np.linspace(0, total_frames - 1, num_frames, dtype=int)
181
+
182
+ for idx in indices:
183
+ cap.set(cv2.CAP_PROP_POS_FRAMES, idx)
184
+ ret, frame = cap.read()
185
+ if ret:
186
+ frame_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
187
+ frames.append(frame_rgb)
188
+
189
+ cap.release()
190
+
191
+ # Process with model
192
+ inputs = processor(frames, return_tensors="pt")
193
+
194
+ with torch.no_grad():
195
+ outputs = model(**inputs)
196
+ predictions = torch.nn.functional.softmax(outputs.logits, dim=-1)
197
+ predicted_class = torch.argmax(predictions, dim=-1).item()
198
+ confidence = predictions[0][predicted_class].item()
199
+
200
+ label = "Violent Crime" if predicted_class == 1 else "Non-Violent"
201
+ return label, confidence
202
+
203
+ # Example usage
204
+ video_path = "path/to/your/video.mp4"
205
+ prediction, confidence = classify_video(video_path)
206
+ print(f"Prediction: {prediction} (Confidence: {confidence:.3f})")
207
+ ```
208
+
209
+ ### Batch Processing
210
+
211
+ ```python
212
+ import os
213
+ from pathlib import Path
214
+
215
+ def process_video_directory(video_dir, output_file="results.txt"):
216
+ results = []
217
+
218
+ for video_file in Path(video_dir).glob("*.mp4"):
219
+ try:
220
+ prediction, confidence = classify_video(str(video_file))
221
+ results.append({
222
+ "file": video_file.name,
223
+ "prediction": prediction,
224
+ "confidence": confidence
225
+ })
226
+ print(f"✅ {video_file.name}: {prediction} ({confidence:.3f})")
227
+ except Exception as e:
228
+ print(f"❌ Error processing {video_file.name}: {e}")
229
+
230
+ # Save results
231
+ with open(output_file, "w") as f:
232
+ for result in results:
233
+ f.write(f"{result['file']}: {result['prediction']} ({result['confidence']:.3f})\n")
234
+
235
+ return results
236
+
237
+ # Process all videos in a directory
238
+ results = process_video_directory("./videos/")
239
+ ```
240
+
241
+ ## 📈 Technical Specifications
242
+
243
+ - **Base Model**: MCG-NJU/videomae-base
244
+ - **Architecture**: Vision Transformer (ViT) adapted for video
245
+ - **Input Resolution**: 224x224 pixels per frame
246
+ - **Temporal Resolution**: 16 frames per video clip
247
+ - **Output Classes**: 2 (Binary classification)
248
+ - **Training Framework**: HuggingFace Transformers
249
+ - **Optimization**: AdamW optimizer with learning rate 5e-5
250
+
251
+ ## ⚠️ Limitations
252
+
253
+ 1. **Dataset Scope**: Trained on a subset of UCF Crime dataset - may not generalize to all types of violence
254
+ 2. **Temporal Context**: Uses 16-frame clips which may miss context in longer sequences
255
+ 3. **Environmental Bias**: Performance may vary with different lighting, camera angles, and video quality
256
+ 4. **False Positives**: May misclassify intense but non-violent activities (sports, action movies)
257
+ 5. **Real-time Performance**: Processing time depends on hardware capabilities
258
+
259
+ ## 🔒 Ethical Considerations
260
+
261
+ ### Intended Use
262
+ - **Primary**: Research and development in video analysis
263
+ - **Secondary**: Security system enhancement with human oversight
264
+ - **Educational**: Computer vision and AI safety research
265
+
266
+ ### Prohibited Uses
267
+ - **Surveillance without consent**: Do not use for unauthorized monitoring
268
+ - **Discriminatory profiling**: Avoid bias against specific groups or communities
269
+ - **Automated punishment**: Never use for automated legal or disciplinary actions
270
+ - **Privacy violation**: Respect privacy laws and individual rights
271
+
272
+ ### Bias and Fairness
273
+ - Model trained on specific dataset that may not represent all populations
274
+ - Regular evaluation needed for bias detection and mitigation
275
+ - Human oversight required for critical applications
276
+ - Consider demographic representation in deployment scenarios
277
+
278
+ ## 📝 Model Card Information
279
+
280
+ - **Developed by**: Research Team
281
+ - **Model Type**: Video Classification (Binary)
282
+ - **Training Data**: UCF Crime Dataset (Subset)
283
+ - **Training Date**: 2025-06-02 11:52:12 UTC
284
+ - **Evaluation Metrics**: Accuracy, Precision, Recall, F1-Score
285
+ - **Intended Users**: Researchers, Security Professionals, Developers
286
+
287
+ ## 📚 Citation
288
+
289
+ If you use this model in your research, please cite:
290
+
291
+ ```bibtex
292
+ @misc{Nikeytas_videomae_crime_detector_ultra_v1,
293
+ title={VideoMAE Fine-tuned for Crime Detection},
294
+ author={Research Team},
295
+ year={2024},
296
+ publisher={Hugging Face},
297
+ url={https://huggingface.co/Nikeytas/videomae-crime-detector-ultra-v1}
298
+ }
299
+ ```
300
+
301
+ ## 🤝 Contributing
302
+
303
+ We welcome contributions to improve the model! Please:
304
+ 1. Report issues with specific examples
305
+ 2. Suggest improvements for bias reduction
306
+ 3. Share evaluation results on new datasets
307
+ 4. Contribute to documentation and examples
308
+
309
+ ## 📞 Contact
310
+
311
+ For questions, issues, or collaboration opportunities, please open an issue in the model repository or contact the development team.
312
 
313
+ ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
314
 
315
+ *Last updated: 2025-06-02 11:52:12 UTC*
316
+ *Model version: 1.0*
317
+ *Framework: HuggingFace Transformers*