Instructions to use D4ve-R/codepilot-mistral-7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use D4ve-R/codepilot-mistral-7b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="D4ve-R/codepilot-mistral-7b")# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("D4ve-R/codepilot-mistral-7b", dtype="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use D4ve-R/codepilot-mistral-7b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "D4ve-R/codepilot-mistral-7b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "D4ve-R/codepilot-mistral-7b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/D4ve-R/codepilot-mistral-7b
- SGLang
How to use D4ve-R/codepilot-mistral-7b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "D4ve-R/codepilot-mistral-7b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "D4ve-R/codepilot-mistral-7b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "D4ve-R/codepilot-mistral-7b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "D4ve-R/codepilot-mistral-7b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use D4ve-R/codepilot-mistral-7b with Docker Model Runner:
docker model run hf.co/D4ve-R/codepilot-mistral-7b
- Model Card for {{ model_id | default("Model ID", true) }}
- Model Details
- Uses
- Bias, Risks, and Limitations
- How to Get Started with the Model
- Training Details
- Evaluation
- Model Examination [optional]
- Environmental Impact
- Technical Specifications [optional]
- Citation [optional]
- Glossary [optional]
- More Information [optional]
- Model Card Authors [optional]
- Model Card Contact
{{ card_data }}
Model Card for {{ model_id | default("Model ID", true) }}
{{ model_summary | default("", true) }}
Model Details
Model Description
{{ model_description | default("", true) }}
- Developed by: {{ developers | default("[More Information Needed]", true)}}
- Funded by [optional]: {{ funded_by | default("[More Information Needed]", true)}}
- Shared by [optional]: {{ shared_by | default("[More Information Needed]", true)}}
- Model type: {{ model_type | default("[More Information Needed]", true)}}
- Language(s) (NLP): {{ language | default("[More Information Needed]", true)}}
- License: {{ license | default("[More Information Needed]", true)}}
- Finetuned from model [optional]: {{ base_model | default("[More Information Needed]", true)}}
Model Sources [optional]
- Repository: {{ repo | default("[More Information Needed]", true)}}
- Paper [optional]: {{ paper | default("[More Information Needed]", true)}}
- Demo [optional]: {{ demo | default("[More Information Needed]", true)}}
Uses
Direct Use
{{ direct_use | default("[More Information Needed]", true)}}
Downstream Use [optional]
{{ downstream_use | default("[More Information Needed]", true)}}
Out-of-Scope Use
{{ out_of_scope_use | default("[More Information Needed]", true)}}
Bias, Risks, and Limitations
{{ bias_risks_limitations | default("[More Information Needed]", true)}}
Recommendations
{{ bias_recommendations | default("Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.", true)}}
How to Get Started with the Model
Use the code below to get started with the model.
{{ get_started_code | default("[More Information Needed]", true)}}
Training Details
Training Data
{{ training_data | default("[More Information Needed]", true)}}
Training Procedure
Preprocessing [optional]
{{ preprocessing | default("[More Information Needed]", true)}}
Training Hyperparameters
- Training regime: {{ training_regime | default("[More Information Needed]", true)}}
Speeds, Sizes, Times [optional]
{{ speeds_sizes_times | default("[More Information Needed]", true)}}
Evaluation
Testing Data, Factors & Metrics
Testing Data
{{ testing_data | default("[More Information Needed]", true)}}
Factors
{{ testing_factors | default("[More Information Needed]", true)}}
Metrics
{{ testing_metrics | default("[More Information Needed]", true)}}
Results
{{ results | default("[More Information Needed]", true)}}
Summary
{{ results_summary | default("", true) }}
Model Examination [optional]
{{ model_examination | default("[More Information Needed]", true)}}
Environmental Impact
Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).
- Hardware Type: {{ hardware | default("[More Information Needed]", true)}}
- Hours used: {{ hours_used | default("[More Information Needed]", true)}}
- Cloud Provider: {{ cloud_provider | default("[More Information Needed]", true)}}
- Compute Region: {{ cloud_region | default("[More Information Needed]", true)}}
- Carbon Emitted: {{ co2_emitted | default("[More Information Needed]", true)}}
Technical Specifications [optional]
Model Architecture and Objective
{{ model_specs | default("[More Information Needed]", true)}}
Compute Infrastructure
{{ compute_infrastructure | default("[More Information Needed]", true)}}
Hardware
{{ hardware | default("[More Information Needed]", true)}}
Software
{{ software | default("[More Information Needed]", true)}}
Citation [optional]
BibTeX:
{{ citation_bibtex | default("[More Information Needed]", true)}}
APA:
{{ citation_apa | default("[More Information Needed]", true)}}
Glossary [optional]
{{ glossary | default("[More Information Needed]", true)}}
More Information [optional]
{{ more_information | default("[More Information Needed]", true)}}
Model Card Authors [optional]
{{ model_card_authors | default("[More Information Needed]", true)}}
Model Card Contact
{{ model_card_contact | default("[More Information Needed]", true)}}
docker model run hf.co/D4ve-R/codepilot-mistral-7b