FredZhang7/one-for-all-toxicity-v3
Link to the distilbert spam defender(https://huggingface.co/FredZhang7/distilbert-spam-defender) Find the v1 (TensorFlow) model in SavedModel format on this page(https://github.com/FredZhang7/tfjs-node-tiny/releases/tag/text-classification). The lice...
Thành tựu nổi bật
Những danh hiệu mới nhất của dự án
#1 Repository Của Tuần
#2 Repository Của Tuần
#1 Repository Của Tuần
Link to the distilbert spam defender
Find the v1 (TensorFlow) model in SavedModel format on this page. The license for the v1 model is Apache 2.0
| v3 | v1 | |
|---|---|---|
| Base Model | bert-base-multilingual-cased | nlpaueb/legal-bert-small-uncased |
| Base Tokenizer | bert-base-multilingual-cased | bert-base-multilingual-cased |
| Framework | PyTorch | TensorFlow |
| Dataset Size | 3.0M | 2.68M |
| Train Split | 80% English 20% English + 100% Multilingual | None |
| English Train Accuracy | 99.5% | N/A (≈97.5%) |
| Other Train Accuracy | 98.6% | 96.6% |
| Final Val Accuracy | 96.8% | 94.6% |
| Languages | 55 | N/A (≈35) |
| Hyperparameters | maxlen=208 padding='max_length' batch_size=112 optimizer=AdamW learning_rate=1e-5 loss=BCEWithLogitsLoss() | maxlen=192 padding='max_length' batch_size=16 optimizer=Adam learning_rate=1e-5 loss="binary_crossentropy" |
| Training Stopped | 7/20/2023 | 9/05/2022 |
I manually annotated more data on top of Toxi Text 3M and added them to the training set. Training on Toxi Text 3M alone results in a biased model that classifies short text with lower precision.
Models tested for v2: roberta, xlm-roberta, bert-small, bert-base-cased/uncased, bert-multilingual-cased/uncased, and alberta-large-v2. Of these, I chose bert-multilingual-cased because it performs better with the same amount of resources as the others for this particular task.
PyTorch
text = "hello world!"
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
tokenizer = AutoTokenizer.from_pretrained("FredZhang7/one-for-all-toxicity-v3")
model = AutoModelForSequenceClassification.from_pretrained("FredZhang7/one-for-all-toxicity-v3").to(device)
encoding = tokenizer.encode_plus(
text,
add_special_tokens=True,
max_length=208,
padding="max_length",
truncation=True,
return_tensors="pt"
)
print('device:', device)
input_ids = encoding["input_ids"].to(device)
attention_mask = encoding["attention_mask"].to(device)
with torch.no_grad():
outputs = model(input_ids, attention_mask=attention_mask)
logits = outputs.logits
predicted_labels = torch.argmax(logits, dim=1)
print(predicted_labels)
Attribution
- If you distribute, remix, adapt, or build upon One-for-all Toxicity v3, please credit "AIstrova Technologies Inc." in your README.md, application description, research, or website.
Similar Projects
Discover other similar projects and AI models.
Whisper Whisper is a state-of-the-art model for automatic speech recognition (ASR) and speech translation, proposed in the paper Robust Speech Recognition via Large-Scale Weak Supervision(https://huggingface.co/papers/2212.04356) by Alec Radford et...
Whisper Whisper is a state-of-the-art model for automatic speech recognition (ASR) and speech translation, proposed in the paper Robust Speech Recognition via Large-Scale Weak Supervision(https://huggingface.co/papers/2212.04356) by Alec Radford et...
Multilingual-E5-small Multilingual E5 Text Embeddings: A Technical Report(https://arxiv.org/pdf/2402.05672). Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, Furu Wei, arXiv 2024 This model has 12 layers and the embedding size is...
Multilingual-E5-large Multilingual E5 Text Embeddings: A Technical Report(https://arxiv.org/pdf/2402.05672). Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, Furu Wei, arXiv 2024 This model has 24 layers and the embedding size is...
XLM-RoBERTa (base-sized model) XLM-RoBERTa model pre-trained on 2.5TB of filtered CommonCrawl data containing 100 languages. It was introduced in the paper Unsupervised Cross-lingual Representation Learning at Scale(https://arxiv.org/abs/1911.02116)...
Multilingual-E5-base Multilingual E5 Text Embeddings: A Technical Report(https://arxiv.org/pdf/2402.05672). Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, Furu Wei, arXiv 2024 This model has 12 layers and the embedding size is 7...
Whisper Whisper is a pre-trained model for automatic speech recognition (ASR) and speech translation. Trained on 680k hours of labelled data, Whisper models demonstrate a strong ability to generalise to many datasets and domains without the need for...
The embedding model trained by Jina AI. jina-embeddings-v3: Multilingual Embeddings With Task LoRA Quick Start Blog(https://jina.ai/news/jina-embeddings-v3-a-frontier-multilingual-embedding-model/parameter-dimensions) | Azure(https://azuremarketplac...
OmniVoice 🌍 OmniVoice is a massively multilingual zero-shot text-to-speech (TTS) model supporting over 600 languages. Built on a novel diffusion language model-style architecture, it delivers high-quality speech w...