teamai-test/.teamai/skills/common/model-support-checker/docs/README.md

model-support-checker

Check whether a HuggingFace / ModelScope model is supported by SGLang, vLLM, or vLLM-Ascend, and since which version.

Setup (one-time)

The tool is stateful: your access mode is persisted in .state/state.json (gitignored), so later runs need no flags or tokens.

On first run you choose between two access modes — local clone is recommended:

	Local clone (recommended)	GitHub PAT
Result quality	Definitive (files grepped on disk)	Best-effort without token; code search 403s anonymously
Rate limits	None	Yes — can break batch/version checks
Repeat checks	Fast (disk reads)	Slower (GitHub API)
Works offline	Yes, after initial clone	No
Cost	~1 GB+ disk (vLLM full history needed)	None
Freshness	Stale until git pull	Always current

python3 main.py --setup local                                              # clone all repos into .state/repos/
python3 main.py --setup local --vllm-path P1 --sglang-path P2 --vllm-ascend-path P3  # reuse existing checkouts
python3 main.py --setup token                                              # GitHub API mode

python3 main.py --doctor        # check setup state and local checkouts
python3 main.py --reset-state   # forget setup (cloned repos are kept and reused)

Notes:

• Local clones are full clones (--depth 1 breaks version detection).
• The GitHub token is NEVER written to .state/; in token mode export GITHUB_TOKEN per run.
• Re-run --setup <mode> anytime to switch modes; explicit --vllm-path / --sglang-path / --vllm-ascend-path / --token flags override the saved state for that run.
• Each run in local mode refreshes checkouts in the background (git fetch) and reports staleness or failures at the end — never blocking the main check.

Methodology

For each framework the checker runs four framework-agnostic steps:

1. Architecture name — read architectures from the model's config.json (HuggingFace first, ModelScope API as fallback). Can be overridden with --arch to skip the network lookup entirely.
2. Official docs (supplementary) — grep the framework's supported-models page.
3. GitHub source (authoritative) — search the framework's models directory on GitHub. All frameworks register a model by mapping the HF architecture string to a class in <models_dir>/<file>.py, so the architecture string is always present in its implementation file.
   • For vLLM this step additionally parses registry.py to extract the model category, module path, class name, and checks _PREVIOUSLY_SUPPORTED_MODELS / _OOT_SUPPORTED_MODELS.
   • For vLLM-Ascend this step parses __init__.py for ModelRegistry.register_model() calls to extract module and class.
4. Version — determine the first framework release that contains the implementation (via the file's earliest commit date → nearest release).

The authoritative "is it supported" check uses the GitHub code-search API with a path filter on the models directory. Anonymous code search returns 403, so a GITHUB_TOKEN is required for a definitive answer. Without a token the tool falls back to scraping the models directory tree + grepping candidate files on the raw CDN; that fallback can be incomplete, so a NO without a token is not definitive. Step 4 also needs a token on rate-limited IPs.

Local checkouts (no GitHub API)

Local mode is the recommended setup (--setup local clones all repos into .state/repos/ automatically). All source reads (models directory, registry.py, and version history) are served from disk via git, and no GitHub API requests are made — only the architecture lookup (step 1) and the docs check (step 2) still touch the network.

# Override saved paths for a single run (no GITHUB_TOKEN needed for source/registry/version)
python3 main.py --framework vllm         --vllm-path         /path/to/vllm         deepseek-ai/DeepSeek-V3
python3 main.py --framework sglang       --sglang-path       /path/to/sglang       Qwen/Qwen3.6-35B-A3B
python3 main.py --framework vllm-ascend  --vllm-ascend-path  /path/to/vllm-ascend  deepseek-ai/DeepSeek-V3

--vllm-ref is ignored when --vllm-path is given. The local path must be a git checkout of the framework repo (so git log/git tag are available for version detection). Keep checkouts fresh with an occasional git pull.

No third-party dependencies — standard library only.

Usage

# After setup, just pass a model id
python3 main.py --framework sglang Qwen/Qwen3.6-35B-A3B
python3 main.py --framework vllm deepseek-ai/DeepSeek-V3
python3 main.py --framework vllm-ascend deepseek-ai/DeepSeek-V3

# Check all frameworks at once (default)
python3 main.py meta-llama/Llama-3.1-8B

# Token mode: GITHUB_TOKEN per run (never stored)
GITHUB_TOKEN=xxx python3 main.py meta-llama/Llama-3.1-8B

# Manually specify architecture name (skip config.json fetch)
python3 main.py --arch LlamaForCausalLM
python3 main.py --framework vllm --arch DeepseekV3ForCausalLM
python3 main.py --framework vllm --arch "DeepseekV3ForCausalLM,DeepseekV2ForCausalLM" some-org/some-model

Options

Flag	Description
model_id	HuggingFace or ModelScope model id (positional; optional when --arch is given).
--setup	One-time setup: local (clone repos into .state/repos/, recommended) or token; persists to .state/state.json.
--reset-state	Forget saved setup state (cloned repos kept).
--doctor	Check setup state and local checkouts, then exit.
--framework	sglang, vllm, vllm-ascend, or all (default: all).
--source	auto, hf, or modelscope for reading config.json.
--arch	Manually specify architecture name(s), comma-separated. Skips config.json fetch entirely.
--token	GitHub token (or set GITHUB_TOKEN env); overrides saved state for this run.
--no-docs	Skip the docs check.
--vllm-ref	vLLM git ref (branch/tag) for the registry check (default: main).
--vllm-path	Local vllm repo checkout; overrides saved path for this run.
--sglang-path	Local sglang repo checkout; overrides saved path for this run.
--vllm-ascend-path	Local vllm-ascend repo checkout; overrides saved path for this run.
-v, --verbose	Verbose output.

License

The Unlicense — released into the public domain.