Instructions to use yanolja/YanoljaNEXT-EEVE-2.8B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use yanolja/YanoljaNEXT-EEVE-2.8B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="yanolja/YanoljaNEXT-EEVE-2.8B", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("yanolja/YanoljaNEXT-EEVE-2.8B", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("yanolja/YanoljaNEXT-EEVE-2.8B", trust_remote_code=True, device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use yanolja/YanoljaNEXT-EEVE-2.8B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "yanolja/YanoljaNEXT-EEVE-2.8B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "yanolja/YanoljaNEXT-EEVE-2.8B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/yanolja/YanoljaNEXT-EEVE-2.8B

SGLang

How to use yanolja/YanoljaNEXT-EEVE-2.8B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "yanolja/YanoljaNEXT-EEVE-2.8B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "yanolja/YanoljaNEXT-EEVE-2.8B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "yanolja/YanoljaNEXT-EEVE-2.8B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "yanolja/YanoljaNEXT-EEVE-2.8B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use yanolja/YanoljaNEXT-EEVE-2.8B with Docker Model Runner:
```
docker model run hf.co/yanolja/YanoljaNEXT-EEVE-2.8B
```

코드 실행하니 이런 에러가 뜨는데 혹시 어떻게 해결하는지 알수있을까요?

by hpho12 - opened Jun 20, 2024

Discussion

hpho12

Jun 20, 2024

/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_token.py:89: UserWarning:
The secret `HF_TOKEN` does not exist in your Colab secrets.
To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.
You will be able to reuse this secret in all of your notebooks.
Please note that authentication is recommended but still optional to access public models or datasets.
warnings.warn(
config.json: 100%
927/927 [00:00<00:00, 15.6kB/s]
Could not locate the configuration_phi.py inside microsoft/phi-2.

HTTPError Traceback (most recent call last)
/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_errors.py in hf_raise_for_status(response, endpoint_name)
303 try:
--> 304 response.raise_for_status()
305 except HTTPError as e:

16 frames
HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/microsoft/phi-2/resolve/main/configuration_phi.py

The above exception was the direct cause of the following exception:

EntryNotFoundError Traceback (most recent call last)
EntryNotFoundError: 404 Client Error. (Request ID: Root=1-6673f775-34a42da434e8c8380aa2799d;8bca6d26-e337-42bd-ac57-4b4713366ef0)

Entry Not Found for url: https://huggingface.co/microsoft/phi-2/resolve/main/configuration_phi.py.

The above exception was the direct cause of the following exception:

OSError Traceback (most recent call last)
/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py in cached_file(path_or_repo_id, filename, cache_dir, force_download, resume_download, proxies, token, revision, local_files_only, subfolder, repo_type, user_agent, _raise_exceptions_for_gated_repo, _raise_exceptions_for_missing_entries, _raise_exceptions_for_connection_errors, _commit_hash, **deprecated_kwargs)
451 if revision is None:
452 revision = "main"
--> 453 raise EnvironmentError(
454 f"{path_or_repo_id} does not appear to have a file named {full_filename}. Checkout "
455 f"'https://huggingface.co/{path_or_repo_id}/tree/{revision}' for available files."

OSError: microsoft/phi-2 does not appear to have a file named configuration_phi.py. Checkout 'https://huggingface.co/microsoft/phi-2/tree/main' for available files.

abcdegh

Apr 23, 2025

model, tokenizer 선언 부분에서 trust_remote_code=True 인자를 제거하시면 실행됩니다.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment