zhezhe/gemma-2-9b-it-Q4_K_M-GGUF
zhezhe/gemma-2-9b-it-Q4KM-GGUF This model was converted to GGUF format from google/gemma-2-9b-it(https://huggingface.co/google/gemma-2-9b-it) using llama.cpp via the ggml.ai's GGUF-my-repo(https://huggingface.co/spaces/ggml-org/gguf-my-repo) space....
Thành tựu nổi bật
Những danh hiệu mới nhất của dự án
#1 Repository Của Tuần
#3 Repository Của Tuần
#3 Repository Của Tuần
zhezhe/gemma-2-9b-it-Q4_K_M-GGUF
This model was converted to GGUF format from google/gemma-2-9b-it using llama.cpp via the ggml.ai's GGUF-my-repo space.
Refer to the original model card for more details on the model.
Use with llama.cpp
Install llama.cpp through brew (works on Mac and Linux)
brew install llama.cpp
Invoke the llama.cpp server or the CLI.
CLI:
llama-cli --hf-repo zhezhe/gemma-2-9b-it-Q4_K_M-GGUF --hf-file gemma-2-9b-it-q4_k_m.gguf -p "The meaning to life and the universe is"
Server:
llama-server --hf-repo zhezhe/gemma-2-9b-it-Q4_K_M-GGUF --hf-file gemma-2-9b-it-q4_k_m.gguf -c 2048
Note: You can also use this checkpoint directly through the usage steps listed in the Llama.cpp repo as well.
Step 1: Clone llama.cpp from GitHub.
git clone https://github.com/ggerganov/llama.cpp
Step 2: Move into the llama.cpp folder and build it with LLAMA_CURL=1 flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).
cd llama.cpp && LLAMA_CURL=1 make
Step 3: Run inference through the main binary.
./llama-cli --hf-repo zhezhe/gemma-2-9b-it-Q4_K_M-GGUF --hf-file gemma-2-9b-it-q4_k_m.gguf -p "The meaning to life and the universe is"
or
./llama-server --hf-repo zhezhe/gemma-2-9b-it-Q4_K_M-GGUF --hf-file gemma-2-9b-it-q4_k_m.gguf -c 2048
Dự án tương tự
Khám phá các dự án và mô hình AI khác có sự tương đồng.
DeepSeek-R1 Paper Link👁️ 1. Introduction We introduce our first-generation reasoning models, DeepSeek-R1-Zero and DeepSeek-R1. DeepSeek-R1-Zero, a model trained via large-scale reinforcement learning (RL) without supervised fine-tuning (SFT) as a p...
Model Information The Meta Llama 3.1 collection of multilingual large language models (LLMs) is a collection of pretrained and instruction tuned generative models in 8B, 70B and 405B sizes (text in/text out). The Llama 3.1 instruction tuned text onl...
Qwen3-0.6B Qwen3 Highlights Qwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models. Built upon extensive training, Qwen3 delivers groundbreaking advancement...
Try gpt-oss · Guides · Model card · OpenAI blog Welcome to the gpt-oss series, OpenAI’s open-weight models(https://openai.com/open-models) designed for powerful reasoning, agentic tasks, and versatile developer use cases. We’re releasing two flavors...
Try gpt-oss · Guides · Model card · OpenAI blog Welcome to the gpt-oss series, OpenAI’s open-weight models(https://openai.com/open-models) designed for powerful reasoning, agentic tasks, and versatile developer use cases. We’re releasing two flavors...
Model Details Meta developed and released the Meta Llama 3 family of large language models (LLMs), a collection of pretrained and instruction tuned generative text models in 8 and 70B sizes. The Llama 3 instruction tuned models are optimized for dia...
GPT-2 Test the whole generation capabilities here: https://transformer.huggingface.co/doc/gpt2-large Pretrained model on English language using a causal language modeling (CLM) objective. It was introduced in this paper(https://d4mucfpksywv.cloudfro...
Model Details Meta developed and released the Meta Llama 3 family of large language models (LLMs), a collection of pretrained and instruction tuned generative text models in 8 and 70B sizes. The Llama 3 instruction tuned models are optimized for dia...
DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence Technical Report👁️ Introduction We present a preview version of DeepSeek-V4 series, including two strong Mixture-of-Experts (MoE) language models — DeepSeek-V4-Pro with 1.6T p...