eunjay commited on
Commit
e4b7fec
·
verified ·
1 Parent(s): ce153ba

docs: clarify MLX usage and tool-call validation

Browse files

Use direct Hugging Face loading commands and document the search-only one-shot smoke-test scope.

Files changed (1) hide show
  1. README.md +34 -12
README.md CHANGED
@@ -1,19 +1,21 @@
1
  ---
2
  license: apache-2.0
 
3
  library_name: mlx
4
  base_model: Alibaba-NLP/WebSailor-3B
5
  base_model_revision: b317a15261674d83d851f0a14761840583bb9dce
6
  tags:
7
- - mlx
8
- - mlx-lm
9
- - 8-bit
10
- - web-agent
 
11
  pipeline_tag: text-generation
12
  ---
13
 
14
  # WebSailor-3B-MLX-8bit-community
15
 
16
- Unofficial community MLX conversion of [Alibaba-NLP/WebSailor-3B](https://huggingface.co/Alibaba-NLP/WebSailor-3B).
17
 
18
  This repository is not an official Alibaba-NLP or Hugging Face release and does not imply endorsement by the original authors.
19
 
@@ -27,36 +29,56 @@ This repository is not an official Alibaba-NLP or Hugging Face release and does
27
  - Conversion tool: `mlx-lm 0.31.2` with `mlx 0.31.1`
28
  - Original license: Apache-2.0; see [`LICENSE`](./LICENSE)
29
 
 
 
 
 
 
 
30
  The conversion command was:
31
 
32
  ```bash
33
  HF_HUB_DISABLE_XET=1 mlx_lm.convert \
34
  --hf-path Alibaba-NLP/WebSailor-3B \
35
  --mlx-path ./WebSailor-3B-MLX-8bit \
36
- --quantize --q-bits 8 --q-group-size 64
 
 
37
  ```
38
 
39
  ## Local usage
40
 
 
 
41
  ```bash
 
 
42
  mlx_lm.generate \
43
- --model ./WebSailor-3B-MLX-8bit \
44
  --prompt "Reply with one short sentence." \
45
  --max-tokens 64
46
  ```
47
 
48
- The conversion was tested locally with `mlx_lm.generate` on Apple Silicon. The model weights alone do not browse the web; web-search or browser tool calls require an external tool/backend and an agent loop.
 
 
 
 
49
 
50
- ## Tool calling with `mlx_lm.server`
51
 
52
- Use the original WebSailor search tool contract: the function name is `search` and `query` is an array of strings. Add an explicit JSON-format instruction to the system message so the model emits a parser-compatible call:
53
 
54
  ```text
55
  If you need the search tool, output exactly one valid JSON object between <tool_call> and </tool_call>. Use double-quoted keys and this shape: {"name":"search","arguments":{"query":["query text"]}}. Never output YAML labels such as name: or arguments:.
56
  ```
57
 
58
- With that instruction, the local OpenAI-compatible MLX server returned `finish_reason: "tool_calls"` and parsed the generated call successfully. The search service itself remains external and must execute the requested queries.
 
 
 
 
59
 
60
  ## Attribution
61
 
62
- Please retain the original model attribution and Apache-2.0 license when redistributing this conversion. This repository contains the converted weights and supporting tokenizer/configuration files; it is not a replacement for the original model card or project code.
 
1
  ---
2
  license: apache-2.0
3
+ language: en
4
  library_name: mlx
5
  base_model: Alibaba-NLP/WebSailor-3B
6
  base_model_revision: b317a15261674d83d851f0a14761840583bb9dce
7
  tags:
8
+ - mlx
9
+ - mlx-lm
10
+ - 8-bit
11
+ - web-agent
12
+ - web-search
13
  pipeline_tag: text-generation
14
  ---
15
 
16
  # WebSailor-3B-MLX-8bit-community
17
 
18
+ Unofficial community MLX conversion of [Alibaba-NLP/WebSailor-3B](https://huggingface.co/Alibaba-NLP/WebSailor-3B) for Apple Silicon.
19
 
20
  This repository is not an official Alibaba-NLP or Hugging Face release and does not imply endorsement by the original authors.
21
 
 
29
  - Conversion tool: `mlx-lm 0.31.2` with `mlx 0.31.1`
30
  - Original license: Apache-2.0; see [`LICENSE`](./LICENSE)
31
 
32
+ Install MLX-LM with:
33
+
34
+ ```bash
35
+ pip install -U mlx-lm
36
+ ```
37
+
38
  The conversion command was:
39
 
40
  ```bash
41
  HF_HUB_DISABLE_XET=1 mlx_lm.convert \
42
  --hf-path Alibaba-NLP/WebSailor-3B \
43
  --mlx-path ./WebSailor-3B-MLX-8bit \
44
+ --quantize \
45
+ --q-bits 8 \
46
+ --q-group-size 64
47
  ```
48
 
49
  ## Local usage
50
 
51
+ MLX-LM can resolve the Hugging Face repository ID directly:
52
+
53
  ```bash
54
+ pip install -U mlx-lm
55
+
56
  mlx_lm.generate \
57
+ --model eunjay/WebSailor-3B-MLX-8bit-community \
58
  --prompt "Reply with one short sentence." \
59
  --max-tokens 64
60
  ```
61
 
62
+ For a local checkout, replace the repository ID with `./WebSailor-3B-MLX-8bit`.
63
+
64
+ The model weights alone do not browse the web; web-search or browser tool calls require an external tool/backend and an agent loop.
65
+
66
+ ## Tool calling and web-search scope
67
 
68
+ The original WebSailor-3B agent harness uses both `search` and `visit` tools through WebAgent. This MLX conversion was validated only with a one-shot `search` tool-call smoke test; the full WebSailor/WebAgent `search` + `visit` harness was not run against this conversion.
69
 
70
+ For the smoke test, an explicit JSON-format instruction was supplied so the model emitted a parser-compatible call:
71
 
72
  ```text
73
  If you need the search tool, output exactly one valid JSON object between <tool_call> and </tool_call>. Use double-quoted keys and this shape: {"name":"search","arguments":{"query":["query text"]}}. Never output YAML labels such as name: or arguments:.
74
  ```
75
 
76
+ The local OpenAI-compatible MLX server parsed that single call successfully. The search service itself remains external and must execute the requested query. This result is not validation of the complete WebSailor search agent or its `visit` flow.
77
+
78
+ ## Compatibility note
79
+
80
+ Hugging Face may display Pi, Hermes, or OpenClaw connection examples automatically for compatible serving endpoints. Those examples indicate endpoint connectivity, not validation of WebSailor's search-agent behavior. Tool-call markers and parsers can vary across MLX-LM versions, so treat the result above as a **one-shot smoke test** and verify the full agent harness before production use.
81
 
82
  ## Attribution
83
 
84
+ Please retain the original model attribution and Apache-2.0 license when redistributing this conversion. This repository contains converted weights and supporting tokenizer/configuration files; it is not a replacement for the [original model card](https://huggingface.co/Alibaba-NLP/WebSailor-3B) or project code.