| --- |
| license: afl-3.0 |
| datasets: |
| - WillHeld/hinglish_top |
| language: |
| - en |
| metrics: |
| - accuracy |
| library_name: transformers |
| pipeline_tag: fill-mask |
| --- |
| |
| ### SRDberta |
|
|
| This is a BERT model trained for Masked Language Modeling for English Data. |
|
|
| ### Dataset |
| Hinglish-Top [Dataset](https://huggingface.co/datasets/WillHeld/hinglish_top) columns |
| - en_query |
| - cs_query |
| - en_parse |
| - cs_parse |
| - domain |
|
|
| ### Training |
| |Epoch|Loss| |
| |:--:|:--:| |
| |1 |0.0485| |
| |2 |0.00837| |
| |3 |0.00812| |
| |4 |0.0029| |
| |5 |0.014| |
| |6 |0.00748| |
| |7 |0.0041| |
| |8 |0.00543| |
| |9 |0.00304| |
| |10 |0.000574| |
|
|
| ### Inference |
| ```python |
| from transformers import AutoTokenizer, AutoModelForMaskedLM, pipeline |
| |
| tokenizer = AutoTokenizer.from_pretrained("SRDdev/SRDBerta") |
| |
| model = AutoModelForMaskedLM.from_pretrained("SRDdev/SRDBerta") |
| |
| fill = pipeline('fill-mask', model='SRDberta', tokenizer='SRDberta') |
| ``` |
| ```python |
| fill_mask = fill.tokenizer.mask_token |
| fill(f'Aap {fill_mask} ho?') |
| ``` |
|
|
| ### Citation |
| Author: @[SRDdev](https://huggingface.co/SRDdev) |
| ``` |
| Name : Shreyas Dixit |
| framework : Pytorch |
| Year: Jan 2023 |
| Pipeline : fill-mask |
| Github : https://github.com/SRDdev |
| LinkedIn : https://www.linkedin.com/in/srddev/ |
| ``` |
