| | --- |
| | library_name: peft |
| | tags: |
| | - code |
| | - instruct |
| | - code-llama |
| | datasets: |
| | - ehartford/dolphin-2.5-mixtral-8x7b |
| | base_model: codellama/CodeLlama-7b-hf |
| | license: apache-2.0 |
| | --- |
| | |
| | ### Finetuning Overview: |
| |
|
| | **Model Used:** codellama/CodeLlama-7b-hf |
| |
|
| | **Dataset:** ehartford/dolphin-2.5-mixtral-8x7b |
| |
|
| | #### Dataset Insights: |
| |
|
| | [No Robots](https://huggingface.co/datasets/HuggingFaceH4/no_robots) is a high-quality dataset of 10,000 instructions and demonstrations created by skilled human annotators. This data can be used for supervised fine-tuning (SFT) to make language models follow instructions better. |
| |
|
| | #### Finetuning Details: |
| |
|
| | With the utilization of [MonsterAPI](https://monsterapi.ai)'s [no-code LLM finetuner](https://monsterapi.ai/finetuning), this finetuning: |
| |
|
| | - Was achieved with great cost-effectiveness. |
| | - Completed in a total duration of 1h 15m 3s for 2 epochs using an A6000 48GB GPU. |
| | - Costed `$2.525` for the entire 2 epochs. |
| |
|
| | #### Hyperparameters & Additional Details: |
| |
|
| | - **Epochs:** 2 |
| | - **Cost Per Epoch:** $1.26 |
| | - **Total Finetuning Cost:** $2.525 |
| | - **Model Path:** codellama/CodeLlama-7b-hf |
| | - **Learning Rate:** 0.0002 |
| | - **Data Split:** 100% train |
| | - **Gradient Accumulation Steps:** 64 |
| | - **lora r:** 64 |
| | - **lora alpha:** 16 |
| |
|
| | --- |
| | license: apache-2.0 |
