Commit c0fd78626 for llama.cpp
commit c0fd78626405c6d9b934efef0ec7291fcc20e3d6
Author: Aleksander Grygier <aleksander.grygier@gmail.com>
Date: Sat Oct 17 17:20:51 2026 +0200
ui : Discover & Download Models from HF Hub (#29584)
* common : resolve <quant>-<sidecar> download tags and list cached sidecars
A Q4_0-mtp style tag now resolves the sidecar file when no model file matches it, so a solo draft or mmproj download actually pulls the file. Cached sidecar files list as their own entries so the state survives a restart, and removing such a tag deletes only the sidecar.
Assisted-by: pi:zai-org/GLM-5.3
* ui : optional sanitized raw HTML in markdown
Add an allowHtml prop to MarkdownContent: raw HTML found in the markdown is
rendered after DOMPurify sanitization instead of being escaped as literal
text. Default stays escaped.
Assisted-by: pi:GLM-5.3-Flash
* ui : add the discover models components
The Discover surface: the curated-catalog list with its search and
skeletons, the details pane with its readme, metadata, Hub stats and
download options, and the standalone download progress bar that replaces
the plain one. The quant download button sizes its select with a new xs
trigger, so the select gains that size and the muted box look. The store
stays incomplete when a repo fails, so the next mount retries it.
Assisted-by: pi:llama.cpp/DeepSeek-V4.1-Flash
* ui : put the discover view in the models manager
The manager dialog gains its second view: the Discover Models call to
action fades the table out, slides the title with its compass mark, and
fades the discover surface in; the arrow button returns to the table.
Assisted-by: pi:llama.cpp/DeepSeek-V4.1-Flash
# Conflicts:
# tools/ui/src/lib/components/ui/button/button.svelte
* ui : add the Enable Discover Models setting
The flag gates browsing and downloading HuggingFace GGUF models, so it is
declared on the discover layer; the repo-tree lookup in the manager layer
above reads it from here.
Assisted-by: pi:llama.cpp/DeepSeek-V4.1-Flash
* ui : resume model downloads from the pause point
A stopped download keeps its partial file instead of deleting it, so
re-posting the tag resumes from disk. The first progress record now
carries the resume offset, the paused snapshot survives into the
resumed entry, and a pause settles as soon as it is requested.
Assisted-by: pi
* ui : lead the discover list with downloads
The discover sidebar lists running downloads above the suggested models,
and steps aside while a query is active. A paused download stays in the
manager table, where it is resumed or dropped.
Assisted-by: pi
* ui : make resuming a paused download work
Re-posting the tag of a paused download hit the router's "already
exists" check, because a paused download parks its entry as DOWNLOADED
until the next reload: a download entry that is not running is now
replaced and resumed. The UI keeps the paused row when the request still
fails, instead of dropping it and looking finished.
Assisted-by: pi
* ui : remember unfinished model downloads
A partial file is named after its content hash, so nothing on disk says
which model it belongs to once the router restarts. The router now keeps
the tags of the downloads it started, clears a tag when the download
finishes or is deleted, and lists what is left as still downloading
without starting anything. The UI adopts those tags as paused on every
list fetch, so a refresh or a fresh open shows an incomplete download and
can resume it from the files already on disk.
Assisted-by: pi
* ui : show a paused download's state on its chip
A paused quant chip reads as paused at rest, the way a running one shows
its spinner, and offers the resume action on hover. The selector's
download row gets a View in Discover action beside its cancel.
Assisted-by: pi
* ui : collapse the download options by default
The copyable download command starts collapsed so the option rows
lead; expanding it stays one click away.
Assisted-by: pi
* ui : keep decision models out of the hub lists
Decision models answer with calibrated probabilities instead of text,
so llama-ui cannot run them: the hub data layer drops them from the
catalog and every search, keyed by the decision-model tag or a
text-classification pipeline.
Assisted-by: pi
* ui : read download tags as quant and sidecar
A `<repo>:<quant>-<sidecar>` tag now parses into its quant and sidecar
instead of one opaque quant token, and ModelsService.draftSidecarsByRepo
collects the router listing's sidecar-only entries per repo, replacing
the inline copy in the models store.
Assisted-by: pi
* ui : cluster draft downloads under their main model
Tracked downloads group by repo: the main weights lead and the draft
sidecars ride inside their item as compact sub-lines, on the selector,
the discover list and the manager table. The main model's controls
pause, resume and cancel the whole cluster, and the manager's download
rows select into the configuration pane.
Assisted-by: pi
* ui : act on tracked downloads from the table rows
The table passes its clusters' draft sub-lines to the model rows. A
row's actions cascade pause and resume, an in-flight row offers Cancel
download while a paused one keeps Delete from disk, and the confirm
dialog's copy follows the state.
Assisted-by: pi
* ui : give downloads a configuration pane of their own
Selecting a tracked download opens the pane in a download mode: live
progress with pause, resume and cancel, and See in Discover, instead of
metadata and load actions that a model without files cannot offer.
Assisted-by: pi
* ui : make the discover details self-fetching and responsive
ModelsDiscoverDetailsContainer owns the details, tree and readme
fetches so hosts outside the discover view can render the pane, and the
details layout adapts to narrow panes with container queries instead of
viewport breakpoints.
Assisted-by: pi
* ui : download a matching draft with the main model
A main quant download now pulls the quant-matched draft of the most
preferred kind the repo ships (DFlash before MTP), fire-and-forget, so
speculative decoding is ready when the main lands. Manual draft chips
stay available; the options row also collapses to one column in narrow
panes.
Assisted-by: pi
* ui : keep discover off by default and couple it with the hub setting
Discover Models starts unchecked. Enabling it turns the Hugging Face
Hub setting on and disabling it turns it back off, so the checkbox
never lies; with discover off the hub setting stays available on its
own for the display data. Stored and reset state are corrected the same
way on load.
Assisted-by: pi
* ui : open the manage models dialog on the manager view
The dialog now opens on the manager instead of returning to the last
view, discover entries still land on the discover view while the
setting allows it, and the discover surfaces follow the setting's
state.
Assisted-by: pi
* ui : give the discover search room for its focus ring
The sticky search bar insets from the panel edges so the focus ring
stays inside the scroll container, and rows scroll behind an opaque bar
instead of through it.
Assisted-by: pi
* common: settle sidecar names on the listing and drop unfinished partials
An infix or trailing sidecar token is ambiguous: draft heads use it
(Model-MTP-BF16.gguf), so do full models with embedded heads
(Model-Q8D-MTP.gguf). Such a name is a sidecar only next to token-free
main weights of the same family, while mmproj or imatrix anywhere in the
name, or a leading token, decides alone (mmproj_f16.gguf).
A quant tag always carries its bit count, so a sidecar named after no
quant (mtp-Model-it.gguf) gets the bare tag: the listing, the plan and
remove all use it. An imatrix tag resolves into a plan slot of its own,
downloaded but never wired into the params.
remove resolves the tag the way a download does and deletes the partial
files of every planned blob, so cancelling a running download or
dropping one left unfinished by an earlier run leaves nothing on disk.
The router passes its token and releases its lock around the call, and
records the unfinished downloads with an atomic write.
llama download succeeds on a sidecar-only plan and prints its paths.
* ui : move dialog views and draft states into enums and constants
The models dialog's views become the ModelsDialogView enum, the draft
subline's states join ModelRowDownloadState, the view fade and the
copy-feedback reset move to ui constants, and the dialog's compass
marks come from DISCOVER_ICON.
Assisted-by: pi
* ui : drop the unused discover surface
The dead openDiscoverModels twin, the commented-out command divider,
the details pane's unused onClose and its header's close button, and a
redundant type cast leave; the command's copy feedback joins the shared
reset constant.
Assisted-by: pi
* ui : trim the download option comments and highlight the chat template
The comment verbosity the options panel and its chips carried shrinks
to the non-obvious why, and the chat template dialog renders through
SyntaxHighlightedCode as jinja instead of a plain pre block.
Assisted-by: pi
* common: only a .gguf file is a sidecar, and mmproj/imatrix as whole words
A non-GGUF file named like a sidecar (quantization-mmproj-Q8_0.json) is
picked as the mmproj, and a sidecar word inside a longer word of a model
name (Phi-2-OmniMatrix.i1-Q4_K_M.gguf) drops the model.
* ui: dedupe draft sidecar badges per quant
The draft sidecar badges deduped by kind and repo without the quant, so
with a draft cached at two quants only one badge survived, and a delete
of the other main quant found no badge to remove its sidecar with.
Also restores the getDownloadEntries doc lost when the clusters method
landed above it.
Assisted-by: pi (zai-org/GLM-5.3-Flash)
* ui: adopt the incomplete download status and drop the paused-ids registry
The store inferred an unfinished download from a downloading entry it
held no local state for, which misread a download running in another
tab as paused and offered a resume the server rejects. The listing
carries the "incomplete" status now, so the adoption keys off it
directly.
The paused ids no longer need to survive in localStorage: the server
remembers them across restarts, and the progress snapshot was stale
after a reload anyway. The stale localStorage key is removed.
Assisted-by: pi (zai-org/GLM-5.3-Flash)
* ui: quant-match the pause and cancel cascades
Pause, resume and cancel cascaded to every draft sidecar of the repo
while delete only took the quant-matched ones, so canceling a Q4_0
download also killed a Q8_0 draft that belongs to another quant's
main. Every cascade acts on the quant-matched drafts now, matching
what a delete takes with it.
Assisted-by: pi (zai-org/GLM-5.3-Flash)
* ui: collapse the duplicated draft subline markup
The downloaded and in-flight branches repeated the identical connector,
label and badge markup; one block with a conditional tail keeps the two
states in sync.
Assisted-by: pi (zai-org/GLM-5.3-Flash)
* ui: scrub positioned styles out of sanitized model card html
The style attribute stays allowed for the KaTeX output, which let a
third party card draw a fixed overlay over the app. The sanitize hook
drops position:fixed and position:absolute values and leaves the KaTeX
layout styles alone.
Assisted-by: pi (zai-org/GLM-5.3-Flash)
* ui: run the discover mount fetch once and reuse the shared repoOf helper
fetch() toggles its loading flag synchronously inside the mount effect,
which re-ran the fetch on every toggle and retried failed catalog repos
endlessly; the effect now untracks the call. The local repoOf copy and
the duplicate services import give way to the shared helpers.
Assisted-by: pi (zai-org/GLM-5.3-Flash)
* ui: narrow the spec-type map to the draft sidecars
SPEC_TYPE carried empty entries for the aux sidecars it is never
indexed with; the map and the command component now type the badge as
a draft sidecar through a draftBadgeOf helper instead of a cast.
Assisted-by: pi (zai-org/GLM-5.3-Flash)
* ui: name the decision-model tags and markdown cache keys
The decision-model hub tag and pipeline tag move into the huggingface
constants next to the other pipeline tags, and the markdown transform
cache prefixes get names instead of bare one-letter tokens.
Assisted-by: pi (zai-org/GLM-5.3-Flash)
* ui: read the discover setting through one truthiness check
Three places read the setting with !== false and one with Boolean();
all four read it the same way now.
Assisted-by: pi (zai-org/GLM-5.3-Flash)
* ui: compare the models dialog view against its enum
The template compared the view state against the 'manage' literal in
three spots while the script used the enum.
Assisted-by: pi (zai-org/GLM-5.3-Flash)
* nit: Remove `+` before draft sidecar badge
* server: list unfinished downloads as incomplete
A download that never finished was reported as "downloading", the same
status a live download carries, so a client could not tell a running
download from a dead one. The tags remembered in unfinished-downloads
now list under a dedicated "incomplete" status, without an instance.
Assisted-by: pi (zai-org/GLM-5.3-Flash)
* server: download a model with its sidecars as -hf does
Resolution goes back to master. POST /models takes the sidecar kinds to
pull with the model (mmproj by default) and the download child gets them
as -hf flags, so the server picks the files. /models lists real models
only, each with the sidecars it has on disk, and incomplete downloads
keep their sidecars across restarts. A stopped download keeps its
partial files for the next attempt, and DELETE purges them through the
same plan the download follows.
* ui: download sidecars as parameters of their model
The download options keep their chips, but a chip no longer posts a
sidecar tag: the store maps it to its model and the sidecar kinds to
pull, the quant chip takes its companion draft in the same download,
and draft badges come from the sidecars each cached model lists.
The server adds the sidecars a cached model lacks when it is posted
again, and DELETE removes a single sidecar of a cached model.
* ui : mirror the server's sidecar lookup in draft badges
The server resolves a sidecar by the case-sensitive `<kind>-` token in the same
directory as the weights. The UI matched the token loosely, so a badge could
offer a draft the server never pulls.
Assisted-by: DeepSeek-V4.1-Flash
* ui : post the model's sidecar kinds with its download
The store keeps one download per model: the chip's kinds ride along with the
model's own sidecars, and a download row no longer expands into one row per
sidecar. Draft badges carry no quant of their own yet, so a sidecar of a cached
model is never labelled with the model's quant.
Assisted-by: DeepSeek-V4.1-Flash
* ui : join a quant's draft into its download chip
One chip per quant, its draft badged inside it and its size covering both
files. The draft is the file the server would pick: the same directory, the
exact quant, else the closest bit depth, so a 4-bit model can pair with a
16-bit head.
Assisted-by: DeepSeek-V4.1-Flash
* ui : drop the draft download rows from the manager and selector
A download is one row per model again: the drafts ride with it, and the rows
read the sidecars on disk off the listing instead of pairing sidecar entries
with their model. Pause, resume and cancel act on the model's own download.
Assisted-by: DeepSeek-V4.1-Flash
* ui : keep the discover download list one row per model
The list leads with one row per downloading model, its sidecars as sub-lines.
A row badges only the draft kinds the server can resolve, so a badge never
outlives the file its lookup would pick.
Assisted-by: DeepSeek-V4.1-Flash
* ui : keep the discover chips readable in dark mode
The metadata chips, the chat template button and the View on Hugging Face link
carried a faint border and no surface over the dialog background, so they were
near invisible. They take the tertiary button surface now, and their icons size
with size-* so the variant's base rule leaves them alone.
Assisted-by: DeepSeek-V4.1-Flash
* ui : carry the draft through the download options
One chip per main quant, its draft badged on it and the chip's downloaded state
covering both files. The drafts sit behind an opt-in checkbox with an inline
type pick, and each row opens its own command: it pins the row's quant and only
names the draft type, so -hfd is gone.
Assisted-by: DeepSeek-V4.1-Flash
* ui : line the details skeleton up with the loaded pane
Same root padding and container context as the loaded view, so the container
queries resolve alike. The download options card pads its blocks instead of
itself, the readme gets its card, and a description line stands in for the
catalog paragraph.
Assisted-by: DeepSeek-V4.1-Flash
* common : drop the partials of a removed sidecar
Removing a sidecar alone left the .downloadInProgress files of the sidecars a
download had planned in blobs/. The removal purges them through the same plan
the download follows, and the test covers a finished draft beside an unfinished
one, whose partial goes while the model keeps its files.
Assisted-by: DeepSeek-V4.1-Flash
* server : name why a download is refused
A model that is already downloading, or that is running, answered "already
exists". It now says which: "is already downloading", or "is running, unload it
first", both before the resumable check. The router test covers the two.
Assisted-by: DeepSeek-V4.1-Flash
* ui : post a download once per click
Two quick clicks on the same chip posted twice, and the second answer was
"already exists". The store tracks the model as pending, returns early while it
is pending or in flight, and reads it as in flight before the server answers, so
the chip shows the download at once. The new client test pins the flows: one
POST per double click, the chip reading downloaded once the list reports the
model, a resume keeping its sidecars, a cancel dropping the whole download, and
a reload restoring the paused state.
Assisted-by: DeepSeek-V4.1-Flash
* ui : keep a plain download row the height of the others
The sub-lines wrapper of a download row was rendered even with nothing in it, so
its padding made the row taller than a plain one and the overlaid progress bar
sat in that empty strip. It renders only with sub-lines now, in the selector and
in the discover list, and the selector row takes the phone padding its
neighbours have.
Assisted-by: DeepSeek-V4.1-Flash
* ui : badge a repo's draft once
The args of a loaded model and the listing of a cached one report the same draft
in different words, one with the file's quant and one with the kind alone, so
the merge kept both and a repo row showed the same kind twice. The badges merge
per kind now, the args one winning as it knows the quant, and the test pins both
sources of a real repo.
Assisted-by: DeepSeek-V4.1-Flash
* server : report the sidecars of a preset or a download on its way
A model the preset also names is registered as a preset, so its drafts were
never reported and no row could badge them. A download on its way reported none
either, though the record of the kinds it pulls is right there: those win over
the plan now, the plan resolves through the preset's hf-repo, and a models dir
stays out of it.
Assisted-by: DeepSeek-V4.1-Flash
* ui : badge a download's drafts and open its model from the row
A draft still coming down badges the model id now, the way a landed one does,
while an on-disk draft keeps its checked sub-line. The row itself opens the
model, the status slot beside it pauses or resumes, and the manager resolves a
focus on a download row so its panel opens.
Assisted-by: DeepSeek-V4.1-Flash
* common : report a removed sidecar that only had a partial
Removing a sidecar alone dropped the result of the partial purge, so a sidecar
that only ever landed as a .downloadInProgress blob answered false and the UI
had no reason to refetch. The purge result is folded into the return now, and
the test covers a partial-only removal.
Assisted-by: DeepSeek-V4.1-Flash
* ui : ignore a pause while the download POST is still on its way
The chip reads as downloading before the server answers, so a quick second click
unloaded a model that had not started and left the store neither in flight nor
paused. A pause returns early while the POST is pending, before it touches the
feed or the paused marker. The client test holds the POST open and pins it.
Assisted-by: DeepSeek-V4.1-Flash
* ui : offer a missing draft as its own chip
A model already on disk without its draft had no way to add it: the quant chip
carried the draft, so it read as a download of both. The quant chip keeps to its
own state now, its joined draft shows only while it rides along or is on disk,
and a downloaded quant missing its draft gets a chip of its own - badge, size,
download - that posts the model with that kind, so the server keeps the files
already there and pulls the draft.
Assisted-by: DeepSeek-V4.1-Flash
* server : list a paused sidecar download by the files on disk
A model on disk whose sidecar download stops keeps its unfinished record,
and the listing reported the kinds that record plans as if they were on
disk: the UI checked a draft that never landed and offered no way to add it.
The planned kinds now stand in only while the download runs.
* chore: Clean up comments
* server : list sidecars only for a model that resolves from a HF repo
A preset on a local file has no HF repo, yet the listing resolved its name
as one: the plan threw on the invalid repo format and GET /models failed
for every model. The sidecars now come from the repo a preset names or a
cached model is listed under, and a local file lists without them.
* common : fix download cancel regression
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
* common : preserve model partials when removing sidecars
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
* server : one sidecar table and an unfinished downloads class
Address review from @ggerganov.
The sidecar kinds live in one table in common/download, mapping each name to
its download option and plan file; the server validates, builds its options
and lists cached sidecars through it instead of its own lists.
The unfinished downloads move into server_unfinished_downloads, a record with
its own mutex and a struct per entry, and server_models exposes them through
unfinished_downloads_get and unfinished_downloads_remove.
* common: one sidecar table, and removals that keep shared files
The resolution, the cached listing and the removal follow a single
table of sidecar kinds, so a kind is declared once with its file
keyword, whether it is a draft, its option and its plan file. The
server reads the same table for the kinds it accepts, the default
sidecars and the child flags.
Removing a tag keeps the files another cached tag still loads, such as
an mmproj named after the removed quant. A sidecar removed on its own
goes for every tag that resolves it.
An unfinished download records the exact paths of its partials when
it starts. A removal deletes those without asking the hub, and keeps a
partial another recorded download also writes. The record is taken
once the child is spawned, and the files go under the models lock, so
a download of the same tag cannot start while they are removed.
* server: drop stale partials and refuse a tag that resolves nothing
Recording a download again deletes the partials of the earlier record
that the new plan no longer writes, such as those of an older hub
revision or of a sidecar the new request leaves out.
A posted tag that resolves no file of the repo is refused before the
download starts. Removing a model follows a single path, whether the
reload erased its entry during the wait or not.
* server: fix the CI race between the router download tests
The pytest workers run in parallel on one shared cache, so a download
test could find the file another test had just fetched and see no
progress, or start from the unfinished downloads record another test
wrote. Each download test now runs on an empty cache of its own.
* server: refuse a DELETE of an unknown name and decide downloads in load()
A DELETE of a name that is neither a model nor a recorded download
went to common_download_remove, and without a tag it wiped the whole
repo cache, including the files of a loaded quant or of a download on
its way. Such a name is now not found, as it was before.
A POST that downloads over an existing entry checked its state outside
the lock, so a load of the same model in between got its instance
killed. The checks now run in load(), under the lock, right before the
spawn. The fake load time of the tests applies to model loads only,
since a download worker loads none.
* server: add incomplete_sidecars to the /models API
A sidecar added to a cached model and then paused kept its record, but
the listing showed only the complete model, so a fresh UI lost the
pause. The model entry now carries incomplete_sidecars, the kinds its
unfinished download has not written yet; a POST with the kind resumes
one and a DELETE with it drops its partial.
* common : drop a kept partial when a resume gets 416
Now that partials survive between attempts, an old or complete
partial makes every retry fail with the same 416.
Delete it so the next attempt can start over from zero.
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
* common: drop a kept partial when a resume gets 200
A server that ignores the range answers 200 to a resume, and the kept
partial then fails every attempt the same way a 416 does. Drop it as
well, so the next attempt starts over without a range.
* server: refuse a download that shares a partial with one on its way
Two downloads of quants that share a file, the mmproj or an mtp picked
by the closest bits, appended to the same partial and renamed a file
with interleaved bytes. A download whose plan writes a partial that a
running download also writes is now refused until that one is done.
* common : check the saved etag before resuming a partial download
Before resuming, compare the etag saved with the partial to the one
the server now serves. If it is missing or different, start over,
so a resumed download never mixes bytes from two versions of the file.
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
---------
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
Co-authored-by: Pascal <admin@serveurperso.com>
Co-authored-by: Adrien Gallouët <angt@huggingface.co>
diff --git a/common/download.cpp b/common/download.cpp
index ae0f7ffed..38f26e1b4 100644
--- a/common/download.cpp
+++ b/common/download.cpp
@@ -39,6 +39,9 @@
// downloader
//
+// the suffix of the file a download writes until it completes
+static const std::string PARTIAL_SUFFIX = ".downloadInProgress";
+
static void write_etag(const std::string & path, const std::string & etag) {
const std::string etag_path = path + ".etag";
fs_write_atomic(std::filesystem::u8path(etag_path), etag);
@@ -186,6 +189,13 @@ static bool common_pull_file(httplib::Client & cli,
[&](const httplib::Response &response) {
if (p.downloaded > 0 && response.status != 206) {
LOG_WRN("%s: server did not respond with 206 Partial Content for a resume request. Status: %d\n", func, response.status);
+ if (response.status == 416 || response.status == 200) {
+ // the kept partial cannot be resumed: 416 for a complete or stale one,
+ // 200 from a server that ignores the range; drop it, so the next
+ // attempt starts over instead of wedging on the same answer
+ ofs.close();
+ remove(path_tmp.c_str());
+ }
return false;
}
if (p.downloaded == 0 && response.status != 200) {
@@ -214,7 +224,7 @@ static bool common_pull_file(httplib::Client & cli,
if (progress_step >= p.total / 1000 || p.downloaded == p.total) {
if (callback) {
callback->on_update(p);
- if (callback->is_cancelled()) {
+ if (callback->is_cancelled() && (p.total == 0 || p.downloaded != p.total)) {
return false;
}
}
@@ -331,10 +341,24 @@ static int common_download_file_single_online(const std::string & url,
}
bool success = false;
- const std::string path_temporary = path + ".downloadInProgress";
+ const std::string path_temporary = path + PARTIAL_SUFFIX;
int delay = retry_delay_seconds;
+ if (!skip_etag && !etag.empty()) {
+ if (std::filesystem::exists(path_temporary) && read_etag(path_temporary) != etag) {
+ LOG_WRN("%s: partial file does not match the server etag, discarding it: %s\n", __func__, path_temporary.c_str());
+ if (remove(path_temporary.c_str()) != 0) {
+ LOG_ERR("%s: unable to delete file: %s\n", __func__, path_temporary.c_str());
+ return -1;
+ }
+ }
+ write_etag(path_temporary, etag);
+ }
+
if (opts.callback) {
+ // report the resume offset, so a resumed file does not start at 0
+ p.downloaded = supports_ranges && std::filesystem::exists(path_temporary) ?
+ std::filesystem::file_size(path_temporary) : 0;
opts.callback->on_start(p);
}
@@ -372,6 +396,7 @@ static int common_download_file_single_online(const std::string & url,
}
if (!etag.empty() && !skip_etag) {
write_etag(path, etag);
+ remove((path_temporary + ".etag").c_str());
}
success = true;
break;
@@ -381,12 +406,7 @@ static int common_download_file_single_online(const std::string & url,
if (opts.callback) {
opts.callback->on_done(p, success);
}
- if (opts.callback && opts.callback->is_cancelled() &&
- std::filesystem::exists(path_temporary)) {
- if (remove(path_temporary.c_str()) != 0) {
- LOG_ERR("%s: unable to delete temporary file: %s\n", __func__, path_temporary.c_str());
- }
- }
+ // a stopped download keeps path_temporary, so the next attempt resumes from it
if (!success) {
LOG_ERR("%s: download failed after %d attempts\n", __func__, max_attempts);
return -1; // max attempts reached
@@ -594,33 +614,65 @@ static hf_cache::hf_file find_best_sibling(const hf_cache::hf_files & files,
return best;
}
-static hf_cache::hf_file find_best_mmproj(const hf_cache::hf_files & files,
- const std::string & model) {
- return find_best_sibling(files, model, "mmproj");
+static const std::vector<common_download_sidecar> sidecars = {
+ { "mmproj", "mmproj", false, &common_download_opts::download_mmproj, &common_download_hf_plan::mmproj },
+ { "mtp", "mtp-", true, &common_download_opts::download_mtp, &common_download_hf_plan::mtp },
+ { "dflash", "dflash-", true, &common_download_opts::download_dflash, &common_download_hf_plan::dflash },
+ { "eagle3", "eagle3-", true, &common_download_opts::download_eagle3, &common_download_hf_plan::eagle3 },
+ { "dspark", "dspark-", true, &common_download_opts::download_dspark, &common_download_hf_plan::dspark },
+};
+
+const std::vector<common_download_sidecar> & common_download_sidecars() {
+ return sidecars;
}
-static hf_cache::hf_file find_best_mtp(const hf_cache::hf_files & files,
- const std::string & model,
- const std::string & tag = "") {
- return find_best_sibling(files, model, "mtp-", tag);
+bool common_download_opts_add_sidecar(common_download_opts & opts, const std::string & kind) {
+ for (const auto & s : sidecars) {
+ if (kind == s.name) {
+ opts.*s.opt = true;
+ return true;
+ }
+ }
+ return false;
}
-static hf_cache::hf_file find_best_eagle3(const hf_cache::hf_files & files,
- const std::string & model,
- const std::string & tag = "") {
- return find_best_sibling(files, model, "eagle3-", tag);
+std::vector<std::string> common_download_plan_sidecars(const common_download_hf_plan & plan) {
+ std::vector<std::string> out;
+ for (const auto & s : sidecars) {
+ if (!(plan.*s.file).path.empty()) {
+ out.emplace_back(s.name);
+ }
+ }
+ return out;
}
-static hf_cache::hf_file find_best_dflash(const hf_cache::hf_files & files,
- const std::string & model,
- const std::string & tag = "") {
- return find_best_sibling(files, model, "dflash-", tag);
+std::vector<common_download_partial> common_download_plan_partials(const common_download_hf_plan & plan) {
+ std::vector<common_download_partial> out;
+
+ // a file in the cache resolves to its snapshot entry, a file still to download to its blob
+ auto add = [&](const std::string & kind, const hf_cache::hf_file & f) {
+ if (!f.local_path.empty() && f.local_path != f.final_path) {
+ out.push_back({ kind, f.local_path + PARTIAL_SUFFIX });
+ }
+ };
+
+ for (const auto & f : plan.model_files) {
+ add("", f);
+ }
+ for (const auto & s : sidecars) {
+ add(s.name, plan.*s.file);
+ }
+ return out;
}
-static hf_cache::hf_file find_best_dspark(const hf_cache::hf_files & files,
- const std::string & model,
- const std::string & tag = "") {
- return find_best_sibling(files, model, "dspark-", tag);
+// true when `name` carries the keyword of a sidecar kind
+static bool name_is_sidecar(const std::string & name) {
+ for (const auto & s : sidecars) {
+ if (name.find(s.keyword) != std::string::npos) {
+ return true;
+ }
+ }
+ return false;
}
static bool gguf_filename_is_model(const std::string & filepath) {
@@ -633,12 +685,7 @@ static bool gguf_filename_is_model(const std::string & filepath) {
filename = filename.substr(pos + 1);
}
- return filename.find("mmproj") == std::string::npos &&
- filename.find("imatrix") == std::string::npos &&
- filename.find("mtp-") == std::string::npos &&
- filename.find("eagle3-") == std::string::npos &&
- filename.find("dflash-") == std::string::npos &&
- filename.find("dspark-") == std::string::npos;
+ return filename.find("imatrix") == std::string::npos && !name_is_sidecar(filename);
}
static hf_cache::hf_file find_best_model(const hf_cache::hf_files & files,
@@ -730,8 +777,11 @@ common_download_hf_plan common_download_get_hf_plan(const common_params_model &
}
} else {
primary = find_best_model(all, tag);
- // a requested sidecar can resolve on its own, without a full model of the same tag
- if (primary.path.empty() && !opts.download_mtp && !opts.download_dflash && !opts.download_eagle3 && !opts.download_dspark) {
+ // a requested draft can resolve on its own, without a full model of the same tag
+ const bool want_draft = std::any_of(sidecars.begin(), sidecars.end(), [&](const common_download_sidecar & s) {
+ return s.draft && opts.*s.opt;
+ });
+ if (primary.path.empty() && !want_draft) {
LOG_ERR("%s: no GGUF files found in repository %s\n", __func__, repo.c_str());
list_available_gguf_files(all);
return plan;
@@ -743,24 +793,22 @@ common_download_hf_plan common_download_get_hf_plan(const common_params_model &
plan.model_files = get_split_files(all, primary);
}
- if (opts.download_mmproj && !primary.path.empty()) {
- plan.mmproj = find_best_mmproj(all, primary.path);
- }
- if (opts.download_mtp) {
- plan.mtp = find_best_mtp(all, primary.path, tag);
- }
- if (opts.download_dflash) {
- plan.dflash = find_best_dflash(all, primary.path, tag);
- }
- if (opts.download_eagle3) {
- plan.eagle3 = find_best_eagle3(all, primary.path, tag);
- }
- if (opts.download_dspark) {
- plan.dspark = find_best_dspark(all, primary.path, tag);
+ // a draft resolves at the tag, the mmproj pairs with the model
+ for (const auto & s : sidecars) {
+ if (!(opts.*s.opt)) {
+ continue;
+ }
+ if (s.draft) {
+ plan.*s.file = find_best_sibling(all, primary.path, s.keyword, tag);
+ } else if (!primary.path.empty()) {
+ plan.*s.file = find_best_sibling(all, primary.path, s.keyword);
+ }
}
- if (primary.path.empty() &&
- plan.mtp.local_path.empty() && plan.dflash.local_path.empty() && plan.eagle3.local_path.empty() && plan.dspark.local_path.empty()) {
+ const bool has_draft = std::any_of(sidecars.begin(), sidecars.end(), [&](const common_download_sidecar & s) {
+ return s.draft && !(plan.*s.file).local_path.empty();
+ });
+ if (primary.path.empty() && !has_draft) {
LOG_ERR("%s: no GGUF files found in repository %s\n", __func__, repo.c_str());
list_available_gguf_files(all);
}
@@ -926,6 +974,11 @@ std::string common_docker_resolve_model(const std::string & docker) {
}
}
+// a cached file that lists a model under its tag: the first split of a model, not a sidecar
+static bool is_listed_model(const gguf_split_info & split) {
+ return split.index == 1 && !split.tag.empty() && !name_is_sidecar(split.prefix);
+}
+
std::vector<common_cached_model_info> common_list_cached_models() {
std::unordered_set<std::string> seen;
std::vector<common_cached_model_info> result;
@@ -934,12 +987,7 @@ std::vector<common_cached_model_info> common_list_cached_models() {
for (const auto & f : files) {
auto split = get_gguf_split_info(f.path);
- if (split.index != 1 || split.tag.empty() ||
- split.prefix.find("mmproj") != std::string::npos ||
- split.prefix.find("mtp-") != std::string::npos ||
- split.prefix.find("eagle3-") != std::string::npos ||
- split.prefix.find("dflash-") != std::string::npos ||
- split.prefix.find("dspark-") != std::string::npos) {
+ if (!is_listed_model(split)) {
continue;
}
if (seen.insert(f.repo_id + ":" + split.tag).second) {
@@ -970,6 +1018,44 @@ std::string common_download_resolve_path(const std::string & hf_repo_with_tag, c
return find_best_model(files, tag).local_path;
}
+// the files -hf loads from the cache, the model with every sidecar, for each cached
+// tag of `repo_id` other than `tag`
+static std::unordered_set<std::string> files_in_use(const std::string & repo_id, const std::string & tag) {
+ common_download_opts opts;
+ opts.offline = true;
+ for (const auto & s : sidecars) {
+ opts.*s.opt = true;
+ }
+
+ std::unordered_set<std::string> tags;
+ std::unordered_set<std::string> used;
+
+ for (const auto & f : hf_cache::get_cached_files(repo_id)) {
+ auto split = get_gguf_split_info(f.path);
+ if (!is_listed_model(split) || split.tag == tag || !tags.insert(split.tag).second) {
+ continue;
+ }
+
+ common_params_model model;
+ model.hf_repo = repo_id + ":" + split.tag;
+
+ auto plan = common_download_get_hf_plan(model, opts);
+
+ for (const auto & m : plan.model_files) {
+ used.insert(m.local_path);
+ }
+ for (const auto & s : sidecars) {
+ const auto & sf = plan.*s.file;
+ if (!sf.local_path.empty()) {
+ used.insert(sf.local_path);
+ }
+ }
+ }
+ return used;
+}
+
+static bool remove_snapshot_entries(const std::string & repo_id, const std::vector<std::filesystem::path> & to_remove);
+
bool common_download_remove(const std::string & hf_repo_with_tag) {
namespace fs = std::filesystem;
@@ -989,11 +1075,13 @@ bool common_download_remove(const std::string & hf_repo_with_tag) {
return false;
}
- // collect snapshot entries whose tag matches
+ // collect snapshot entries whose tag matches, but a file another tag loads stays
+ const auto used = files_in_use(repo_id, tag_upper);
+
std::vector<fs::path> to_remove;
for (const auto & f : files) {
auto split = get_gguf_split_info(f.path);
- if (split.tag == tag_upper) {
+ if (split.tag == tag_upper && used.find(f.local_path) == used.end()) {
to_remove.emplace_back(f.local_path);
}
}
@@ -1002,6 +1090,13 @@ bool common_download_remove(const std::string & hf_repo_with_tag) {
return false;
}
+ return remove_snapshot_entries(repo_id, to_remove);
+}
+
+// remove snapshot entries of `repo_id`, then the blobs no remaining entry references
+static bool remove_snapshot_entries(const std::string & repo_id, const std::vector<std::filesystem::path> & to_remove) {
+ namespace fs = std::filesystem;
+
// resolve blob paths from symlinks before deleting snapshot entries
std::vector<fs::path> blobs_to_check;
for (const auto & p : to_remove) {
@@ -1053,3 +1148,26 @@ bool common_download_remove(const std::string & hf_repo_with_tag) {
return true;
}
+
+bool common_download_remove_sidecars(const std::string & hf_repo_with_tag, const common_download_opts & opts) {
+ common_params_model model;
+ model.hf_repo = hf_repo_with_tag;
+
+ common_download_opts cached = opts;
+ cached.offline = true;
+
+ auto plan = common_download_get_hf_plan(model, cached);
+
+ std::vector<std::filesystem::path> to_remove;
+ for (const auto & s : sidecars) {
+ const auto & f = plan.*s.file;
+ if (!f.local_path.empty()) {
+ to_remove.emplace_back(std::filesystem::u8path(f.local_path));
+ }
+ }
+ if (to_remove.empty()) {
+ return false;
+ }
+
+ return remove_snapshot_entries(common_download_split_repo_tag(hf_repo_with_tag).first, to_remove);
+}
diff --git a/common/download.h b/common/download.h
index 8c30cfc3e..4dfd0c983 100644
--- a/common/download.h
+++ b/common/download.h
@@ -104,10 +104,17 @@ std::string common_docker_resolve_model(const std::string & docker);
// Remove a cached model from disk
// input format: "user/model" or "user/model:tag"
// - if tag is omitted, removes the entire repo cache directory
-// - if tag is present, removes only files matching that tag (and orphaned blobs)
+// - if tag is present, removes only files matching that tag (and orphaned blobs),
+// except the files another cached tag of the repo still loads
// returns true if anything was removed
bool common_download_remove(const std::string & hf_repo_with_tag);
+// remove the cached sidecar files `opts` selects for "user/model:tag", as the cached
+// plan resolves them, and leave the model files in place; a sidecar file another tag
+// shares goes for every tag that resolves it
+// returns true if anything was removed
+bool common_download_remove_sidecars(const std::string & hf_repo_with_tag, const common_download_opts & opts);
+
struct common_download_hf_plan {
hf_cache::hf_file primary;
hf_cache::hf_files model_files;
@@ -119,3 +126,33 @@ struct common_download_hf_plan {
hf_cache::hf_file preset; // if set, only this file is downloaded
};
common_download_hf_plan common_download_get_hf_plan(const common_params_model & model, const common_download_opts & opts);
+
+// a sidecar kind a download can pull next to its model: mmproj, and the draft heads
+// of --spec-type draft-<name>
+struct common_download_sidecar {
+ const char * name; // the kind, as the API names it
+ const char * keyword; // the part of a file name that marks a file of this kind
+ bool draft; // a draft head, resolved at the tag even without a model
+
+ bool common_download_opts::* opt; // the option that requests it
+ hf_cache::hf_file common_download_hf_plan::* file; // the plan file it resolves to
+};
+
+// every sidecar kind, the single list the resolution, the listing and the removal follow
+const std::vector<common_download_sidecar> & common_download_sidecars();
+
+// select the sidecar `kind` in `opts`; returns false for an unknown kind
+bool common_download_opts_add_sidecar(common_download_opts & opts, const std::string & kind);
+
+// the sidecar kinds `plan` resolves a file for
+std::vector<std::string> common_download_plan_sidecars(const common_download_hf_plan & plan);
+
+// a file `plan` still has to download, at the path its partial takes while it is
+// incomplete; kind is the sidecar it belongs to, empty for the model files
+struct common_download_partial {
+ std::string kind;
+ std::string path;
+};
+
+// the partials the download of `plan` writes, for the files not in the cache yet
+std::vector<common_download_partial> common_download_plan_partials(const common_download_hf_plan & plan);
diff --git a/tests/test-model-resolution.cpp b/tests/test-model-resolution.cpp
index 5c793e24a..eb7b30936 100644
--- a/tests/test-model-resolution.cpp
+++ b/tests/test-model-resolution.cpp
@@ -15,6 +15,7 @@
#include <cstdio>
#include <cstdlib>
#include <filesystem>
+#include <fstream>
#include <map>
#include <thread>
#include <string>
@@ -339,6 +340,124 @@ static void test_plan_resolution() {
g_repos.clear();
}
+// put a file in the cache the way a finished download leaves it, refs/main included
+static void cache_put(const std::string & repo_id, const std::string & path) {
+ namespace fs = std::filesystem;
+
+ const fs::path local = cached(repo_id, path);
+ fs::create_directories(local.parent_path());
+ { std::ofstream(local) << "gguf"; }
+
+ const fs::path ref = local.parent_path().parent_path().parent_path() / "refs" / "main";
+ fs::create_directories(ref.parent_path());
+ if (!fs::exists(ref)) {
+ std::ofstream(ref) << COMMIT << "\n";
+ }
+}
+
+//
+// partials of a download: the files of its plan not in the cache yet, next to
+// the blobs, named after the oids the listing serves
+//
+
+static void test_plan_partials() {
+ namespace fs = std::filesystem;
+
+ printf("test-model-resolution: plan partials\n");
+
+ g_context = "plan partials";
+ g_repos.clear();
+ g_repos["test/partial"] = { "model-Q4_0.gguf", "mmproj-model-F16.gguf", "mtp-model-Q4_0.gguf" };
+
+ // the mmproj is complete, the model and its mtp are still to download
+ cache_put("test/partial", "mmproj-model-F16.gguf");
+
+ common_download_opts opts;
+ opts.download_mmproj = true;
+ opts.download_mtp = true;
+
+ const auto parts = common_download_plan_partials(common_download_get_hf_plan(model_ref("test/partial:Q4_0"), opts));
+
+ const fs::path blobs = cache_dir / "models--test--partial" / "blobs";
+ REQUIRE(parts.size() == 2);
+ REQUIRE(parts[0].kind.empty());
+ REQUIRE(fs::path(parts[0].path).lexically_normal() == (blobs / "0000000000000000000000000000000000000001.downloadInProgress").lexically_normal());
+ REQUIRE_EQ(parts[1].kind, "mtp");
+ REQUIRE(fs::path(parts[1].path).lexically_normal() == (blobs / "0000000000000000000000000000000000000003.downloadInProgress").lexically_normal());
+
+ fs::remove_all(cache_dir / "models--test--partial");
+ g_repos.clear();
+}
+
+//
+// removal of a sidecar alone: its snapshot entry goes, the model keeps its files
+//
+
+static void test_sidecar_removal() {
+ namespace fs = std::filesystem;
+
+ printf("test-model-resolution: sidecar removal\n");
+
+ g_context = "sidecar removal";
+ g_repos.clear();
+
+ cache_put("test/sidecar", "model-Q4_0.gguf");
+ cache_put("test/sidecar", "dflash-model-Q4_0.gguf");
+ cache_put("test/sidecar", "mtp-model-Q4_0.gguf");
+
+ common_download_opts opts;
+ opts.download_dflash = true;
+
+ // the selected sidecar goes, the other one and the model stay
+ REQUIRE(common_download_remove_sidecars("test/sidecar:Q4_0", opts));
+ REQUIRE(!fs::exists(cached("test/sidecar", "dflash-model-Q4_0.gguf")));
+ REQUIRE(fs::exists(cached("test/sidecar", "mtp-model-Q4_0.gguf")));
+ REQUIRE(fs::exists(cached("test/sidecar", "model-Q4_0.gguf")));
+ REQUIRE(!common_download_remove_sidecars("test/sidecar:Q4_0", opts));
+
+ fs::remove_all(cache_dir / "models--test--sidecar");
+}
+
+//
+// removal of a tag keeps the files another cached tag still loads: a sidecar
+// named after the removed tag that another quant resolves stays on disk
+//
+
+static void test_shared_removal() {
+ namespace fs = std::filesystem;
+
+ printf("test-model-resolution: shared removal\n");
+
+ g_context = "shared removal";
+ g_repos.clear();
+
+ for (const char * f : { "model-Q8_0.gguf", "model-BF16.gguf", "mmproj-model-BF16.gguf", "mtp-model-BF16.gguf" }) {
+ cache_put("test/shared", f);
+ }
+
+ // the BF16 model goes, the mmproj and the mtp Q8_0 resolves stay
+ REQUIRE(common_download_remove("test/shared:BF16"));
+ REQUIRE(!fs::exists(cached("test/shared", "model-BF16.gguf")));
+ REQUIRE(fs::exists(cached("test/shared", "mmproj-model-BF16.gguf")));
+ REQUIRE(fs::exists(cached("test/shared", "mtp-model-BF16.gguf")));
+ REQUIRE(fs::exists(cached("test/shared", "model-Q8_0.gguf")));
+
+ fs::remove_all(cache_dir / "models--test--shared");
+
+ // with no other tag, the files of the removed tag all go, sidecars included
+ for (const char * f : { "model-BF16.gguf", "mmproj-model-BF16.gguf", "mtp-model-BF16.gguf" }) {
+ cache_put("test/alone", f);
+ }
+
+ REQUIRE(common_download_remove("test/alone:BF16"));
+ REQUIRE(!fs::exists(cached("test/alone", "model-BF16.gguf")));
+ REQUIRE(!fs::exists(cached("test/alone", "mmproj-model-BF16.gguf")));
+ REQUIRE(!fs::exists(cached("test/alone", "mtp-model-BF16.gguf")));
+ REQUIRE(!common_download_remove("test/alone:BF16"));
+
+ fs::remove_all(cache_dir / "models--test--alone");
+}
+
//
// end-to-end assembly: real CLI parsing, real handler init resolving over the
// loopback, downloads skipped by flipping offline before apply
@@ -509,6 +628,9 @@ int main(void) {
common_set_env("MODEL_ENDPOINT", "http://127.0.0.1:" + std::to_string(port) + "/");
test_plan_resolution();
+ test_plan_partials();
+ test_sidecar_removal();
+ test_shared_removal();
test_task_assembly();
server.stop();
diff --git a/tools/server/README.md b/tools/server/README.md
index c42336c7f..ec82db007 100644
--- a/tools/server/README.md
+++ b/tools/server/README.md
@@ -2106,6 +2106,32 @@ Note: for "downloading" state, there can be multiple files be downloading in par
}
```
+A download that never finished (paused or interrupted) is listed under a separate
+"incomplete" status, without an instance: posting the model to `/models` again resumes it
+from the files already on disk, and `DELETE /models` drops its partial files.
+
+```json
+"status": {
+ "value": "incomplete",
+ "args": []
+}
+```
+
+Cached models and incomplete downloads carry the sidecar kinds they have on disk
+(`mmproj`, `mtp`, `dflash`, `eagle3`, `dspark`), resolved the way `-hf` loads them:
+
+```json
+"sidecars": ["mmproj", "mtp"]
+```
+
+A cached model whose download of more sidecars never finished also lists the kinds not on
+disk yet: posting the model with them resumes them, and `DELETE /models` with `&sidecar={kind}`
+drops their partial files.
+
+```json
+"incomplete_sidecars": ["mtp"]
+```
+
### POST `/models/load`: Load a model
Load a model
@@ -2225,7 +2251,7 @@ Example events:
Trigger a new download (non-blocking), the progress can be tracked via SSE endpoint `/models/sse`
-To cancel model downloading, send an event to `/models/unload`
+To pause a download, send an event to `/models/unload`: it keeps its partial files and is listed as "incomplete" until it is posted again. To cancel it and drop its files, use `DELETE /models`.
Download procedure:
- Send POST request to `/models`
@@ -2238,9 +2264,14 @@ Payload:
```json
{
"model": "ggml-org/gemma-3-4b-it-GGUF:Q4_K_M",
+ "sidecars": ["mmproj", "mtp"]
}
```
+`sidecars` lists the sidecar kinds pulled with the model, picked by the server the way `-hf` picks them: `mmproj`, and the draft heads `mtp`, `dflash`, `eagle3` and `dspark`. It defaults to `["mmproj"]`, as `-hf` does.
+
+Posting a model that is already in the cache and not running downloads the sidecars it lacks and keeps the files already on disk.
+
Response (download is started in the background):
```json
@@ -2267,6 +2298,10 @@ IMPORTANT: only model stored in cache can be deleted. You cannot delete models i
Model name must be passed via query param: `?model={name}`
+To remove one sidecar of a cached model and keep the model, add `&sidecar={kind}` (`mmproj`, `mtp`, `dflash`, `eagle3` or `dspark`); the model must not be running.
+
+Deleting a model keeps the files another cached quant of the repo still loads, such as an mmproj named after the deleted quant, and the partial files another download still writes. A sidecar deleted on its own goes for every quant that shares it.
+
If delete success, it will send an SSE event of type `model_remove`
Response:
diff --git a/tools/server/server-models.cpp b/tools/server/server-models.cpp
index 60353240a..d77837c86 100644
--- a/tools/server/server-models.cpp
+++ b/tools/server/server-models.cpp
@@ -26,6 +26,7 @@
#include <chrono>
#include <queue>
#include <filesystem>
+#include <fstream>
#include <random>
#include <sstream>
#include <cstring>
@@ -618,6 +619,7 @@ server_models::server_models(
LOG_WRN("failed to get server executable path: %s\n", e.what());
LOG_WRN("using original argv[0] as fallback: %s\n", argv[0]);
}
+ unfinished_downloads.load();
load_models();
debug_fake_timing = !common_get_env("LLAMA_SERVER_DEBUG_FAKE_TIMING").empty();
}
@@ -686,6 +688,233 @@ void server_models::add_model(server_model_meta && meta) {
};
}
+static bool has_sidecar(const std::vector<std::string> & sidecars, const std::string & kind) {
+ return std::find(sidecars.begin(), sidecars.end(), kind) != sidecars.end();
+}
+
+// reject a sidecar kind no download can pull
+static void check_sidecar(const std::string & kind) {
+ std::vector<std::string> kinds;
+ for (const auto & s : common_download_sidecars()) {
+ kinds.emplace_back(s.name);
+ }
+ if (!has_sidecar(kinds, kind)) {
+ throw std::invalid_argument("unknown sidecar '" + kind + "', expected one of " + string_join(kinds, ", "));
+ }
+}
+
+// the sidecars a download asks for; by default the ones -hf pulls without a draft type
+static std::vector<std::string> parse_sidecars(const json & body) {
+ std::vector<std::string> defaults;
+ for (const auto & s : common_download_sidecars()) {
+ if (!s.draft) {
+ defaults.emplace_back(s.name);
+ }
+ }
+ std::vector<std::string> sidecars = json_value(body, "sidecars", defaults);
+ for (const auto & kind : sidecars) {
+ check_sidecar(kind);
+ }
+ return sidecars;
+}
+
+// the download options of a model with its sidecars, the plan a removal purges partials from
+static common_download_opts sidecar_download_opts(const common_params & params, const std::vector<std::string> & sidecars) {
+ common_download_opts opts;
+ opts.bearer_token = params.hf_token;
+ for (const auto & kind : sidecars) {
+ common_download_opts_add_sidecar(opts, kind);
+ }
+ return opts;
+}
+
+// the HF repo a model resolves from: a preset names its own, a cached model is
+// listed under it, and a local file or a models dir entry has none
+static std::string model_hf_repo(const server_model_meta & meta) {
+ std::string hf_repo;
+ if (meta.preset.get_option("LLAMA_ARG_HF_REPO", hf_repo) && !hf_repo.empty()) {
+ return hf_repo;
+ }
+ return meta.source == SERVER_MODEL_SOURCE_CACHE ? meta.name : "";
+}
+
+// the sidecar kinds of a HF repo present on disk, as the offline plan resolves them
+static json cached_sidecars(const std::string & hf_repo) {
+ common_params_model model;
+ model.hf_repo = hf_repo;
+
+ common_download_opts opts;
+ opts.offline = true;
+ for (const auto & s : common_download_sidecars()) {
+ opts.*s.opt = true;
+ }
+
+ return common_download_plan_sidecars(common_download_get_hf_plan(model, opts));
+}
+
+// replace any inherited value of `key` in a child environment
+static void set_child_env(std::vector<std::string> & env, const std::string & key, const std::string & value) {
+ env.erase(std::remove_if(env.begin(), env.end(), [&](const std::string & e) {
+ return e.rfind(key + "=", 0) == 0;
+ }), env.end());
+ env.push_back(key + "=" + value);
+}
+
+// where the unfinished downloads are remembered, inside the model cache
+static std::string unfinished_downloads_path() {
+ return hf_cache::get_cache_path() + "/unfinished-downloads.json";
+}
+
+void server_unfinished_downloads::load() {
+ std::ifstream in(unfinished_downloads_path());
+ if (!in) {
+ return;
+ }
+ std::string raw((std::istreambuf_iterator<char>(in)), std::istreambuf_iterator<char>());
+ std::lock_guard<std::mutex> lk(mutex);
+ try {
+ json data = json::parse(raw);
+ for (const auto & d : data.value("downloads", json::array())) {
+ server_unfinished_download download;
+ download.model = json_value(d, "model", std::string());
+ download.sidecars = json_value(d, "sidecars", std::vector<std::string>());
+ for (const auto & p : d.value("partials", json::array())) {
+ download.partials.push_back({ json_value(p, "kind", std::string()), json_value(p, "path", std::string()) });
+ }
+ if (!download.model.empty()) {
+ downloads.push_back(std::move(download));
+ }
+ }
+ } catch (const std::exception & e) {
+ LOG_WRN("failed to read unfinished downloads: %s\n", e.what());
+ }
+}
+
+void server_unfinished_downloads::save() {
+ json data;
+ data["downloads"] = json::array();
+ for (const auto & d : downloads) {
+ json partials = json::array();
+ for (const auto & p : d.partials) {
+ partials.push_back({ {"kind", p.kind}, {"path", p.path} });
+ }
+ data["downloads"].push_back({ {"model", d.model}, {"sidecars", d.sidecars}, {"partials", partials} });
+ }
+ try {
+ fs_write_atomic(std::filesystem::u8path(unfinished_downloads_path()), data.dump());
+ } catch (const std::exception & e) {
+ LOG_WRN("failed to record unfinished downloads: %s\n", e.what());
+ }
+}
+
+void server_unfinished_downloads::delete_partials(const std::vector<common_download_partial> & parts) {
+ for (const auto & p : parts) {
+ const bool shared = std::any_of(downloads.begin(), downloads.end(), [&](const server_unfinished_download & d) {
+ return std::any_of(d.partials.begin(), d.partials.end(), [&](const common_download_partial & q) {
+ return q.path == p.path;
+ });
+ });
+ if (shared) {
+ continue;
+ }
+ std::error_code ec;
+ std::filesystem::remove(std::filesystem::u8path(p.path), ec);
+ }
+}
+
+void server_unfinished_downloads::add(const server_unfinished_download & download) {
+ std::lock_guard<std::mutex> lk(mutex);
+ std::vector<common_download_partial> stale;
+ auto it = std::find_if(downloads.begin(), downloads.end(), [&](const auto & d) { return d.model == download.model; });
+ if (it != downloads.end()) {
+ stale = std::move(it->partials);
+ *it = download;
+ } else {
+ downloads.push_back(download);
+ }
+ save();
+ delete_partials(stale);
+}
+
+void server_unfinished_downloads::forget(const std::string & model) {
+ std::lock_guard<std::mutex> lk(mutex);
+ auto it = std::find_if(downloads.begin(), downloads.end(), [&](const auto & d) { return d.model == model; });
+ if (it != downloads.end()) {
+ downloads.erase(it);
+ save();
+ }
+}
+
+bool server_unfinished_downloads::remove(const std::string & model) {
+ std::lock_guard<std::mutex> lk(mutex);
+ auto it = std::find_if(downloads.begin(), downloads.end(), [&](const auto & d) { return d.model == model; });
+ if (it == downloads.end()) {
+ return false;
+ }
+ auto partials = std::move(it->partials);
+ downloads.erase(it);
+ save();
+ delete_partials(partials);
+ return true;
+}
+
+bool server_unfinished_downloads::remove_sidecar(const std::string & model, const std::string & kind) {
+ std::lock_guard<std::mutex> lk(mutex);
+ auto it = std::find_if(downloads.begin(), downloads.end(), [&](const auto & d) { return d.model == model; });
+ if (it == downloads.end() || !has_sidecar(it->sidecars, kind)) {
+ return false;
+ }
+ it->sidecars.erase(std::remove(it->sidecars.begin(), it->sidecars.end(), kind), it->sidecars.end());
+
+ std::vector<common_download_partial> dropped;
+ auto & partials = it->partials;
+ auto pos = std::stable_partition(partials.begin(), partials.end(), [&](const common_download_partial & p) {
+ return p.kind != kind;
+ });
+ dropped.assign(pos, partials.end());
+ partials.erase(pos, partials.end());
+
+ save();
+ delete_partials(dropped);
+ return true;
+}
+
+std::vector<server_unfinished_download> server_unfinished_downloads::get_all() {
+ std::lock_guard<std::mutex> lk(mutex);
+ return downloads;
+}
+
+std::vector<server_unfinished_download> server_models::unfinished_downloads_get() {
+ return unfinished_downloads.get_all();
+}
+
+void server_models::unfinished_downloads_remove(const std::string & name) {
+ // under the lock, so a download of the same tag cannot start while its files go
+ std::lock_guard<std::mutex> lk(mutex);
+ if (mapping.find(name) != mapping.end()) {
+ throw std::invalid_argument("model '" + name + "' is downloading");
+ }
+ // a name with no record is no model: its files belong to the cached tags of the repo
+ if (!unfinished_downloads.remove(name)) {
+ throw std::runtime_error("model name=" + name + " is not found");
+ }
+ bool ok = common_download_remove(name);
+ SRV_INF("removing unfinished download name=%s (%s)\n", name.c_str(), ok ? "succeeded" : "nothing cached");
+}
+
+bool server_models::remove_sidecar(const std::string & name, const std::string & kind) {
+ std::lock_guard<std::mutex> lk(mutex);
+ auto it = mapping.find(name);
+ if (it != mapping.end() && (it->second.meta.is_running() || it->second.meta.status == SERVER_MODEL_STATUS_DOWNLOADING)) {
+ throw std::invalid_argument("model '" + name + "' is in use, unload it first");
+ }
+ // a sidecar that only ever landed as a partial is still something removed
+ bool ok = unfinished_downloads.remove_sidecar(name, kind);
+ ok = common_download_remove_sidecars(name, sidecar_download_opts(base_params, { kind })) || ok;
+ SRV_INF("removing sidecar %s of model name=%s (%s)\n", kind.c_str(), name.c_str(), ok ? "succeeded" : "nothing cached");
+ return ok;
+}
+
void server_models::notify_sse(const std::string & event, const std::string & model_id, const json & data) {
std::unique_ptr<server_task_result_router> result = std::make_unique<server_task_result_router>();
result->data = {
@@ -1135,7 +1364,8 @@ void server_models::load(const std::string & name) {
}
void server_models::load(const std::string & name, const load_options & opts) {
- if (debug_fake_timing) {
+ // the fake load time of a model; a download worker loads none
+ if (debug_fake_timing && opts.mode == SERVER_CHILD_MODE_NORMAL) {
// do not hold the mutex here, other requests must keep making progress
std::this_thread::sleep_for(std::chrono::seconds(2));
}
@@ -1152,6 +1382,43 @@ void server_models::load(const std::string & name, const load_options & opts) {
// against the freshest preset and a consistent mapping state
cv.wait(lk, [this]() { return !is_reloading; });
+ // a download over an existing entry is decided under the lock, so a load of the
+ // same model cannot start in between: a paused download or an idle cached model
+ // is downloaded again with its files kept, a download on its way and a running
+ // model are refused, and a preset or local model is not a download
+ if (opts.mode == SERVER_CHILD_MODE_DOWNLOAD) {
+ auto it = mapping.find(name);
+ if (it != mapping.end()) {
+ const auto & cur = it->second.meta;
+ if (cur.status == SERVER_MODEL_STATUS_DOWNLOADING) {
+ throw std::invalid_argument("model '" + name + "' is already downloading");
+ }
+ if (cur.is_running()) {
+ throw std::invalid_argument("model '" + name + "' is running, unload it first");
+ }
+ if (cur.source != SERVER_MODEL_SOURCE_CACHE) {
+ throw std::invalid_argument("model '" + name + "' already exists");
+ }
+ SRV_INF("resuming download for model '%s'\n", name.c_str());
+ }
+
+ // two downloads writing the same partial would interleave their bytes
+ for (const auto & d : unfinished_downloads.get_all()) {
+ auto other = mapping.find(d.model);
+ if (d.model == name || other == mapping.end() || other->second.meta.status != SERVER_MODEL_STATUS_DOWNLOADING) {
+ continue;
+ }
+ for (const auto & p : d.partials) {
+ const bool shared = std::any_of(opts.partials.begin(), opts.partials.end(), [&](const common_download_partial & q) {
+ return q.path == p.path;
+ });
+ if (shared) {
+ throw std::invalid_argument("model '" + name + "' shares files with the download of '" + d.model + "' on its way, retry once it is done");
+ }
+ }
+ }
+ }
+
auto meta = opts.custom_meta.has_value() ? *opts.custom_meta : mapping[name].meta;
if (meta.status != SERVER_MODEL_STATUS_UNLOADED) {
SRV_INF("model %s is not ready\n", name.c_str());
@@ -1201,6 +1468,23 @@ void server_models::load(const std::string & name, const load_options & opts) {
inst.meta.status = SERVER_MODEL_STATUS_DOWNLOADING;
child_env.push_back("LLAMA_SERVER_CHILD_MODE=download");
child_env.push_back("LLAMA_ARG_HF_REPO=" + name);
+
+ // the sidecars as -hf would pull them: the draft types, and the mmproj switch
+ // for the one that is not a draft
+ bool mmproj = false;
+ std::string spec_types;
+ for (const auto & s : common_download_sidecars()) {
+ if (!has_sidecar(opts.sidecars, s.name)) {
+ continue;
+ }
+ if (s.draft) {
+ spec_types += (spec_types.empty() ? "" : ",") + std::string("draft-") + s.name;
+ } else {
+ mmproj = true;
+ }
+ }
+ set_child_env(child_env, "LLAMA_ARG_MMPROJ_AUTO", mmproj ? "1" : "0");
+ set_child_env(child_env, "LLAMA_ARG_SPEC_TYPE", spec_types.empty() ? "none" : spec_types);
}
SRV_INF("%s", "spawning server instance with args:\n");
@@ -1216,6 +1500,11 @@ void server_models::load(const std::string & name, const load_options & opts) {
}
}
+ // remember the download: the partial files carry a content hash, not the name
+ if (opts.mode == SERVER_CHILD_MODE_DOWNLOAD) {
+ unfinished_downloads.add({ name, opts.sidecars, opts.partials });
+ }
+
// old process should have exited already, but just in case, we clean it up here
{
auto it = mapping.find(name);
@@ -1390,6 +1679,12 @@ void server_models::update_download_progress(const std::string & name, const com
// mark the instance to be erased on next load_models() call
it->second.meta.status = SERVER_MODEL_STATUS_DOWNLOADED;
need_reload = true;
+
+ // a download that failed keeps its tag: its partial files are still
+ // there and re-posting the tag resumes them
+ if (ok) {
+ unfinished_downloads.forget(name);
+ }
} else {
json & info = it->second.meta.loaded_info;
if (!info.contains("progress")) {
@@ -1445,24 +1740,14 @@ bool server_models::remove(const std::string & name) {
|| meta.status == SERVER_MODEL_STATUS_DOWNLOADED;
});
- // re-find after wait - load_models() may have erased the entry during the wait
- it = mapping.find(name);
- if (it == mapping.end()) {
- // load_models() already erased the entry; we just need to clean up the cached files on disk
- lk.unlock();
- bool ok = common_download_remove(name);
- SRV_INF("removing model name=%s from cache (%s)\n", name.c_str(), ok ? "succeeded" : "partial");
- notify_sse("model_remove", name, {});
- return true;
- }
+ // load_models() may have erased the entry during the wait; the files go under the
+ // lock, so a download of the same tag cannot start meanwhile
+ mapping.erase(name);
+ unfinished_downloads.remove(name);
- // remove from disk (best-effort: cancelled downloads may have no cached files)
+ // best-effort: a cancelled download may have no cached files
bool ok = common_download_remove(name);
- mapping.erase(name);
- if (!ok) {
- SRV_WRN("removing model name=%s from disk returned false (no cached files?)\n", name.c_str());
- }
- SRV_INF("removing model name=%s from cache (%s)\n", name.c_str(), ok ? "succeeded" : "partial");
+ SRV_INF("removing model name=%s from cache (%s)\n", name.c_str(), ok ? "succeeded" : "nothing cached");
notify_sse("model_remove", name, {});
return true;
}
@@ -2091,6 +2376,9 @@ void server_models_routes::init_routes() {
json models_json = json::array();
auto all_models = models.get_all_meta();
std::time_t t = std::time(0);
+ // a download on its way has no files to resolve yet, only the kinds it pulls
+ const auto unfinished = models.unfinished_downloads_get();
+
for (const auto & meta : all_models) {
if (meta.hidden) {
continue; // cache model deduplicated by a preset
@@ -2132,6 +2420,35 @@ void server_models_routes::init_routes() {
model_info["context_length"] = meta.n_ctx_train;
}
+ // the sidecars a model has on disk, resolved as -hf would load them, for a
+ // model that resolves from a HF repo. A download on its way reports the
+ // kinds it pulls, as an unfinished one does
+ const auto it_unfinished = std::find_if(unfinished.begin(), unfinished.end(),
+ [&](const server_unfinished_download & d) { return d.model == meta.name; });
+ const std::string hf_repo = model_hf_repo(meta);
+
+ if (it_unfinished != unfinished.end() && meta.status == SERVER_MODEL_STATUS_DOWNLOADING) {
+ model_info["sidecars"] = it_unfinished->sidecars;
+ } else if (!hf_repo.empty()) {
+ const json cached = cached_sidecars(hf_repo);
+ model_info["sidecars"] = cached;
+
+ // the sidecars of an unfinished download of this model that are not on
+ // disk yet: a POST with the kind resumes one, a DELETE with it drops it
+ if (it_unfinished != unfinished.end()) {
+ std::vector<std::string> incomplete;
+ for (const auto & p : it_unfinished->partials) {
+ if (!p.kind.empty() && !has_sidecar(incomplete, p.kind) &&
+ std::find(cached.begin(), cached.end(), p.kind) == cached.end()) {
+ incomplete.push_back(p.kind);
+ }
+ }
+ if (!incomplete.empty()) {
+ model_info["incomplete_sidecars"] = incomplete;
+ }
+ }
+ }
+
// merge with loaded_info from the child process if available
if (meta.is_running()) {
for (auto it = meta.loaded_info.begin(); it != meta.loaded_info.end(); ++it) {
@@ -2142,6 +2459,33 @@ void server_models_routes::init_routes() {
}
models_json.push_back(model_info);
}
+
+ const json architecture_default = json {
+ {"input_modalities", json::array({"text"})},
+ {"output_modalities", json::array({"text"})},
+ };
+
+ // a download that never finished has no instance: report it as incomplete so
+ // the UI can offer to resume it, without starting anything
+ for (const auto & d : unfinished) {
+ if (models.has_model(d.model)) {
+ continue; // complete, or downloading right now
+ }
+ models_json.push_back(json {
+ {"id", d.model},
+ {"aliases", json::array()},
+ {"tags", json::array()},
+ {"object", "model"},
+ {"owned_by", "llamacpp"},
+ {"created", t},
+ {"status", json { {"value", server_model_status_to_string(SERVER_MODEL_STATUS_INCOMPLETE)}, {"args", json::array()} }},
+ {"architecture", architecture_default},
+ {"source", server_model_source_to_string(SERVER_MODEL_SOURCE_CACHE)},
+ {"can_remove", true},
+ {"sidecars", d.sidecars},
+ });
+ }
+
res_ok(res, {
{"data", models_json},
{"object", "list"},
@@ -2193,32 +2537,27 @@ void server_models_routes::init_routes() {
if (name.empty()) {
throw std::invalid_argument("model must be a non-empty string");
}
+ const auto sidecars = parse_sidecars(body);
- common_params p;
- p.model.hf_repo = name;
- p.hf_token = params.hf_token;
+ common_params_model model;
+ model.hf_repo = name;
- // validate by fetching metadata
- bool ok = false;
+ // validate by fetching metadata, through the plan the download follows
+ common_download_hf_plan plan;
try {
- common_models_handler_init(p, LLAMA_EXAMPLE_SERVER);
- ok = true;
+ plan = common_download_get_hf_plan(model, sidecar_download_opts(params, sidecars));
} catch (...) {
SRV_ERR("unknown error while validating model '%s'\n", name.c_str());
// other exceptions will be handled by the outer ex_wrapper()
throw;
}
- if (!ok) {
- throw std::invalid_argument("model validation failed, unable to download");
- }
-
- // reject if model already exists
- if (models.has_model(name)) {
- throw std::invalid_argument("model '" + name + "' already exists");
+ // a tag that resolves no file has nothing to download
+ if (plan.preset.path.empty() && plan.primary.path.empty() && common_download_plan_sidecars(plan).empty()) {
+ throw std::invalid_argument("model '" + name + "' resolves no file to download");
}
- // then, proceed with the actual download
+ // then, proceed with the actual download; load() decides over an existing entry
SRV_INF("starting download for model '%s'\n", name.c_str());
{
server_models::load_options load_opts;
@@ -2226,6 +2565,8 @@ void server_models_routes::init_routes() {
load_opts.custom_meta = server_model_meta{};
load_opts.custom_meta->source = SERVER_MODEL_SOURCE_CACHE;
load_opts.custom_meta->name = name;
+ load_opts.sidecars = sidecars;
+ load_opts.partials = common_download_plan_partials(plan);
models.load(name, load_opts);
}
@@ -2241,6 +2582,22 @@ void server_models_routes::init_routes() {
throw std::invalid_argument("model must be a non-empty string");
}
+ // a sidecar alone goes, the model keeps its files
+ std::string sidecar = req.get_param("sidecar");
+ if (!sidecar.empty()) {
+ check_sidecar(sidecar);
+ res_ok(res, {{"success", models.remove_sidecar(name, sidecar)}});
+ return res;
+ }
+
+ // a download listed from an earlier run has no instance, only its tag and
+ // its partial files
+ if (!models.has_model(name)) {
+ models.unfinished_downloads_remove(name);
+ res_ok(res, {{"success", true}});
+ return res;
+ }
+
models.remove(name); // throws on error
res_ok(res, {{"success", true}});
diff --git a/tools/server/server-models.h b/tools/server/server-models.h
index f415b7c5f..b04fc7384 100644
--- a/tools/server/server-models.h
+++ b/tools/server/server-models.h
@@ -16,6 +16,7 @@
#include <set>
#include <string>
#include <unordered_map>
+#include <vector>
/**
* state diagram:
@@ -31,6 +32,7 @@
enum server_model_status {
// TODO: also add downloading state when the logic is added
SERVER_MODEL_STATUS_DOWNLOADING,
+ SERVER_MODEL_STATUS_INCOMPLETE, // a download that never finished, listed without an instance
SERVER_MODEL_STATUS_DOWNLOADED,
SERVER_MODEL_STATUS_UNLOADED,
SERVER_MODEL_STATUS_LOADING,
@@ -52,6 +54,7 @@ enum server_child_mode {
static std::string server_model_status_to_string(server_model_status status) {
switch (status) {
case SERVER_MODEL_STATUS_DOWNLOADING: return "downloading";
+ case SERVER_MODEL_STATUS_INCOMPLETE: return "incomplete";
case SERVER_MODEL_STATUS_DOWNLOADED: return "downloaded";
case SERVER_MODEL_STATUS_UNLOADED: return "unloaded";
case SERVER_MODEL_STATUS_LOADING: return "loading";
@@ -108,6 +111,51 @@ struct server_model_meta {
void update_caps(const common_params & base);
};
+// a download that never finished: the tag it was asked under, the sidecar kinds it
+// pulls and the partial files its plan writes
+struct server_unfinished_download {
+ std::string model;
+ std::vector<std::string> sidecars;
+ std::vector<common_download_partial> partials;
+};
+
+// the downloads that never finished, recorded next to the model cache. The cache names
+// partial files after their content hash, so this record is the only trace of what is
+// still incomplete: it lets a restarted router list, resume or remove them
+// a partial another recorded download also writes is never deleted
+// these functions are thread-safe
+struct server_unfinished_downloads {
+ // read the record an earlier run left
+ void load();
+
+ // record a download that starts, replacing an earlier record of the same model,
+ // and delete the partials of that record the new plan no longer writes
+ void add(const server_unfinished_download & download);
+
+ // forget a download whose files are complete
+ void forget(const std::string & model);
+
+ // forget a download and delete its partials; returns false when it is not recorded
+ bool remove(const std::string & model);
+
+ // drop the sidecar `kind` from a download and delete its partials; returns false
+ // when the download is not recorded with that kind
+ bool remove_sidecar(const std::string & model, const std::string & kind);
+
+ // every recorded download
+ std::vector<server_unfinished_download> get_all();
+
+private:
+ std::mutex mutex;
+ std::vector<server_unfinished_download> downloads;
+
+ // write the record back; callers hold the mutex
+ void save();
+
+ // delete the partials of `parts` no recorded download writes; callers hold the mutex
+ void delete_partials(const std::vector<common_download_partial> & parts);
+};
+
struct server_models_routes;
struct server_lru_sched; // defined in server-models.cpp
struct server_monitor; // defined in server-models.cpp
@@ -134,6 +182,9 @@ private:
// models asked to stop, still counted as running until the monitor records their exit
std::set<std::string> stopping_models;
+ // downloads that never finished, with the sidecar kinds they pull
+ server_unfinished_downloads unfinished_downloads;
+
// set to true while load_models() is executing a reload; load() will wait until clear
bool is_reloading = false;
@@ -252,6 +303,16 @@ public:
// check if a model instance exists (thread-safe)
bool has_model(const std::string & name);
+ // downloads that never finished, with their sidecar kinds (thread-safe)
+ std::vector<server_unfinished_download> unfinished_downloads_get();
+
+ // drop a download that will never be resumed, its record and its partial files;
+ // throws for a name that is not recorded (thread-safe)
+ void unfinished_downloads_remove(const std::string & name);
+
+ // remove the sidecar `kind` of a model that is not in use, its files and partials (thread-safe)
+ bool remove_sidecar(const std::string & name, const std::string & kind);
+
// return a copy of model metadata (thread-safe)
std::optional<server_model_meta> get_meta(const std::string & name);
@@ -262,6 +323,10 @@ public:
server_child_mode mode = SERVER_CHILD_MODE_NORMAL;
// used for spawning a downloading child process
std::optional<server_model_meta> custom_meta = std::nullopt;
+ // download mode: the sidecar kinds pulled with the model, as -hf would, and
+ // the partial files the download plan writes
+ std::vector<std::string> sidecars;
+ std::vector<common_download_partial> partials;
};
// load and unload model instances
diff --git a/tools/server/tests/unit/test_router.py b/tools/server/tests/unit/test_router.py
index 5355ab736..6df41f8d6 100644
--- a/tools/server/tests/unit/test_router.py
+++ b/tools/server/tests/unit/test_router.py
@@ -1,3 +1,4 @@
+import shutil
import threading
import pytest
from utils import *
@@ -546,6 +547,15 @@ def _wait_for_sse_event(collected: list, event_type: str, model: str, timeout: i
return False
+def _own_cache(name: str) -> str:
+ """An empty model cache for one test: the workers run in parallel, and a download
+ test must not see the files or the record another test writes in the shared cache."""
+ path = os.path.join(TMP_DIR, name)
+ shutil.rmtree(path, ignore_errors=True)
+ os.makedirs(path)
+ return path
+
+
def test_router_download_model():
"""Case 1: download a model at the model limit, verify SSE events and GET /models."""
global server
@@ -626,3 +636,230 @@ def test_router_delete_model():
# Model should no longer appear in GET /models
ids = _get_model_ids(is_reload=False)
assert MODEL_DOWNLOAD_ID not in ids, f"{MODEL_DOWNLOAD_ID} still present after deletion"
+
+
+def test_router_download_sidecars():
+ """Case 3: sidecars are a parameter of the download, never models of their own."""
+ global server
+ server.llama_cache = _own_cache("cache_download_sidecars")
+ server.start()
+
+ server.make_request("DELETE", f"/models?model={MODEL_DOWNLOAD_ID}")
+
+ # an unknown sidecar kind is rejected before anything starts
+ res = server.make_request("POST", "/models", data={"model": MODEL_DOWNLOAD_ID, "sidecars": ["lora"]})
+ assert res.status_code == 400
+ assert MODEL_DOWNLOAD_ID not in _get_model_ids(is_reload=False)
+
+ sse_events: list = []
+ stop = threading.Event()
+ sse_ready = threading.Event()
+ threading.Thread(
+ target=_listen_sse, args=(server, sse_events, stop, sse_ready), daemon=True
+ ).start()
+ assert sse_ready.wait(10), "SSE client failed to connect"
+
+ res = server.make_request("POST", "/models", data={"model": MODEL_DOWNLOAD_ID, "sidecars": []})
+ assert res.status_code == 200
+ finished = _wait_for_sse_event(
+ sse_events, "download_finished", MODEL_DOWNLOAD_ID, MODEL_DOWNLOAD_TIMEOUT
+ )
+ stop.set()
+ assert finished, "Never received download_finished SSE event"
+
+ # the model lists the sidecars it has on disk, and no sidecar is listed as a model
+ res = server.make_request("GET", "/models")
+ assert res.status_code == 200
+ entries = {item["id"]: item for item in res.body.get("data", [])}
+ assert entries[MODEL_DOWNLOAD_ID]["sidecars"] == []
+ for kind in ("mmproj", "mtp", "dflash", "eagle3", "dspark"):
+ assert not any(i.endswith("-" + kind) for i in entries), f"a {kind} sidecar is listed as a model"
+
+ server.make_request("DELETE", f"/models?model={MODEL_DOWNLOAD_ID}")
+
+
+def test_router_download_conflicts():
+ """A download is refused over one on its way, and over a model that is running."""
+ global server
+ server.models_max = 2
+ server.llama_cache = _own_cache("cache_download_conflicts")
+ server.start()
+
+ server.make_request("DELETE", f"/models?model={MODEL_DOWNLOAD_ID}")
+
+ # a download on its way is not started again; the entry reports the state before
+ # the child fetches anything, so the probe lands inside the download
+ res = server.make_request("POST", "/models", data={"model": MODEL_DOWNLOAD_ID})
+ assert res.status_code == 200
+ _wait_for_model_status(MODEL_DOWNLOAD_ID, {"downloading"}, timeout=30)
+
+ res = server.make_request("POST", "/models", data={"model": MODEL_DOWNLOAD_ID})
+ assert res.status_code == 400
+ assert "is already downloading" in res.body["error"]["message"]
+
+ # a loaded model holds its files, so it goes only once it is unloaded; the
+ # reload that follows the download parks the entry as unloaded
+ _wait_for_model_status(MODEL_DOWNLOAD_ID, {"downloaded", "unloaded"}, timeout=MODEL_DOWNLOAD_TIMEOUT)
+ _load_model_and_wait(MODEL_DOWNLOAD_ID, timeout=120)
+
+ res = server.make_request("POST", "/models", data={"model": MODEL_DOWNLOAD_ID})
+ assert res.status_code == 400
+ assert "is running, unload it first" in res.body["error"]["message"]
+
+
+def test_router_delete_unknown_name():
+ """A name that is neither a model nor a recorded download is not found, and the
+ cached tags of its repo keep their files."""
+ global server
+ server.llama_cache = _own_cache("cache_delete_unknown_name")
+ server.start()
+
+ res = server.make_request("POST", "/models", data={"model": MODEL_DOWNLOAD_ID, "sidecars": []})
+ assert res.status_code == 200
+ _wait_for_model_status(MODEL_DOWNLOAD_ID, {"downloaded", "unloaded"}, timeout=MODEL_DOWNLOAD_TIMEOUT)
+
+ repo = MODEL_DOWNLOAD_ID.split(":")[0]
+ res = server.make_request("DELETE", f"/models?model={repo}")
+ assert res.status_code != 200
+ assert "is not found" in res.body["error"]["message"]
+ assert MODEL_DOWNLOAD_ID in _get_model_ids(is_reload=True)
+
+ server.make_request("DELETE", f"/models?model={MODEL_DOWNLOAD_ID}")
+
+
+def test_router_incomplete_sidecars():
+ """A cached model lists the sidecars its unfinished download has not written yet, until
+ they are dropped."""
+ global server
+ cache_dir = _own_cache("cache_incomplete_sidecars")
+ server.llama_cache = cache_dir
+ server.start()
+
+ res = server.make_request("POST", "/models", data={"model": MODEL_DOWNLOAD_ID, "sidecars": []})
+ assert res.status_code == 200
+ _wait_for_model_status(MODEL_DOWNLOAD_ID, {"downloaded", "unloaded"}, timeout=MODEL_DOWNLOAD_TIMEOUT)
+ server.stop()
+
+ # a paused mtp of the cached model, as the record of an earlier run holds it
+ partial_path = os.path.abspath(os.path.join(cache_dir, "test-mtp.gguf.downloadInProgress"))
+ with open(partial_path, "w") as f:
+ f.write("partial")
+ with open(os.path.join(cache_dir, "unfinished-downloads.json"), "w") as f:
+ json.dump({"downloads": [{
+ "model": MODEL_DOWNLOAD_ID,
+ "sidecars": ["mtp"],
+ "partials": [{"kind": "mtp", "path": partial_path}],
+ }]}, f)
+
+ server.start()
+
+ def entry() -> dict:
+ res = server.make_request("GET", "/models")
+ assert res.status_code == 200
+ return {item["id"]: item for item in res.body.get("data", [])}[MODEL_DOWNLOAD_ID]
+
+ assert entry()["sidecars"] == []
+ assert entry()["incomplete_sidecars"] == ["mtp"]
+
+ res = server.make_request("DELETE", f"/models?model={MODEL_DOWNLOAD_ID}&sidecar=mtp")
+ assert res.status_code == 200
+ assert res.body.get("success") is True
+ assert not os.path.exists(partial_path)
+ assert "incomplete_sidecars" not in entry()
+
+ server.make_request("DELETE", f"/models?model={MODEL_DOWNLOAD_ID}")
+
+
+def test_router_preset_model_sidecars():
+ """A preset model reports the sidecars its files have on disk, like a cached one."""
+ global server
+
+ preset_path = os.path.join(TMP_DIR, "test_preset_sidecars.ini")
+
+ with open(preset_path, "w") as f:
+ f.write("[preset-sidecars]\nhf-repo = ggml-org/tinygemma3-GGUF:Q8_0\n")
+
+ server.models_preset = preset_path
+ server.start()
+
+ try:
+ # the mmproj of that repo lands next to the model, as -hf would pull it
+ res = server.make_request("POST", "/models", data={"model": MODEL_A, "sidecars": ["mmproj"]})
+ assert res.status_code == 200
+ _wait_for_model_status(MODEL_A, {"downloaded", "unloaded"}, timeout=MODEL_DOWNLOAD_TIMEOUT)
+
+ res = server.make_request("GET", "/models")
+ assert res.status_code == 200
+ entries = {item["id"]: item for item in res.body.get("data", [])}
+
+ assert entries["preset-sidecars"]["source"] == "preset"
+ assert entries["preset-sidecars"]["sidecars"] == ["mmproj"]
+ finally:
+ os.remove(preset_path)
+
+
+def test_router_local_preset_listing():
+ """A preset on a local file resolves from no HF repo: it lists without sidecars."""
+ global server
+
+ model_path = os.path.join(TMP_DIR, "test_local_preset.gguf")
+ preset_path = os.path.join(TMP_DIR, "test_local_preset.ini")
+
+ with open(model_path, "w") as f:
+ f.write("GGUF")
+ with open(preset_path, "w") as f:
+ f.write(f"[local]\nmodel = {model_path}\n")
+
+ server.models_preset = preset_path
+ server.start()
+
+ try:
+ res = server.make_request("GET", "/models")
+ assert res.status_code == 200
+ entries = {item["id"]: item for item in res.body.get("data", [])}
+
+ assert entries["local"]["source"] == "preset"
+ assert "sidecars" not in entries["local"]
+ finally:
+ os.remove(preset_path)
+ os.remove(model_path)
+
+
+def test_router_download_unresolved_tag():
+ """A tag that resolves no file of the repo is refused before anything starts."""
+ global server
+ server.start()
+
+ model = MODEL_DOWNLOAD_ID.split(":")[0] + ":Q9_Z"
+ res = server.make_request("POST", "/models", data={"model": model})
+ assert res.status_code == 400
+ assert "resolves no file to download" in res.body["error"]["message"]
+ assert model not in _get_model_ids(is_reload=False)
+
+
+def test_router_download_drops_stale_partials():
+ """Posting a recorded download again deletes the partials its new plan no longer writes."""
+ global server
+
+ cache_dir = _own_cache("cache_download_drops_stale_partials")
+ stale_path = os.path.abspath(os.path.join(cache_dir, "test-stale.gguf.downloadInProgress"))
+ record_path = os.path.join(cache_dir, "unfinished-downloads.json")
+
+ with open(stale_path, "w") as f:
+ f.write("stale")
+ with open(record_path, "w") as f:
+ json.dump({"downloads": [{
+ "model": MODEL_DOWNLOAD_ID,
+ "sidecars": ["mtp"],
+ "partials": [{"kind": "mtp", "path": stale_path}],
+ }]}, f)
+
+ server.llama_cache = cache_dir
+ server.start()
+
+ res = server.make_request("POST", "/models", data={"model": MODEL_DOWNLOAD_ID, "sidecars": []})
+ assert res.status_code == 200
+ assert not os.path.exists(stale_path), "the stale partial survived the new record"
+
+ _wait_for_model_status(MODEL_DOWNLOAD_ID, {"downloaded", "unloaded"}, timeout=MODEL_DOWNLOAD_TIMEOUT)
+ server.make_request("DELETE", f"/models?model={MODEL_DOWNLOAD_ID}")
diff --git a/tools/server/tests/utils.py b/tools/server/tests/utils.py
index 1c84f5121..288452808 100644
--- a/tools/server/tests/utils.py
+++ b/tools/server/tests/utils.py
@@ -127,6 +127,7 @@ class ServerProcess:
mcp_servers_config: str | None = None
mcp_servers_json: str | None = None
cors_origins: str | None = None
+ llama_cache: str | None = None
# session variables
process: subprocess.Popen | None = None
@@ -145,7 +146,9 @@ class ServerProcess:
**os.environ,
"LLAMA_SERVER_DEBUG_FAKE_TIMING": "1",
}
- if "LLAMA_CACHE" not in os.environ:
+ if self.llama_cache is not None:
+ env["LLAMA_CACHE"] = self.llama_cache
+ elif "LLAMA_CACHE" not in os.environ:
env["LLAMA_CACHE"] = "tmp"
if self.external_server:
print(f"[external_server]: Assuming external server running on {self.server_host}:{self.server_port}")
diff --git a/tools/ui/src/lib/components/app/content/MarkdownContent/MarkdownContent.svelte b/tools/ui/src/lib/components/app/content/MarkdownContent/MarkdownContent.svelte
index c217a769a..895e9a80a 100644
--- a/tools/ui/src/lib/components/app/content/MarkdownContent/MarkdownContent.svelte
+++ b/tools/ui/src/lib/components/app/content/MarkdownContent/MarkdownContent.svelte
@@ -7,6 +7,7 @@
getMdastNodeHash,
isAppendMode
} from './markdown-utils';
+ import { SAFE_HTML_CONFIG } from './safe-html-config';
import {
ActionIconCopyToClipboard,
CodeBlockActions,
@@ -44,6 +45,7 @@
import { detectIncompleteCodeBlock, highlightCode, type IncompleteCodeBlock } from '$lib/utils';
import { sanitizeSvg } from '$lib/utils/sanitize-svg';
import { mountSvgShadow } from '$lib/utils/svg-shadow';
+ import DOMPurify from 'dompurify';
import type { Root as HastRoot, RootContent as HastRootContent } from 'hast';
import githubLightCss from 'highlight.js/styles/github.css?inline';
import githubDarkCss from 'highlight.js/styles/github-dark.css?inline';
@@ -57,6 +59,8 @@
content: string;
class?: string;
disableMath?: boolean;
+ /** Render raw HTML found in the markdown (sanitized) instead of escaping it. */
+ allowHtml?: boolean;
}
interface MarkdownBlock {
@@ -65,7 +69,13 @@
contentHash?: string;
}
- let { attachments, class: className = '', content, disableMath = false }: Props = $props();
+ let {
+ allowHtml = false,
+ attachments,
+ class: className = '',
+ content,
+ disableMath = false
+ }: Props = $props();
let containerRef = $state<HTMLDivElement>();
let renderedBlocks = $state<MarkdownBlock[]>([]);
@@ -121,6 +131,8 @@
// Per-instance transform cache, avoids re-transforming stable blocks during streaming
// Garbage collected when component is destroyed (on conversation change)
+ const CACHE_KEY_PLAIN = 'plain';
+ const CACHE_KEY_ALLOW_HTML = 'allow-html';
const transformCache = new SvelteMap<string, string>();
let previousContent = '';
@@ -183,7 +195,10 @@
index: number
): Promise<{ html: string; hash: string }> {
const hash = getMdastNodeHash(node, index);
- const cached = transformCache.get(hash);
+ // the rendered HTML also depends on allowHtml (sanitized raw vs escaped),
+ // so the cache is keyed per mode
+ const cacheKey = `${allowHtml ? CACHE_KEY_ALLOW_HTML : CACHE_KEY_PLAIN}:${hash}`;
+ const cached = transformCache.get(cacheKey);
if (cached) {
return { hash, html: cached };
@@ -192,10 +207,13 @@
const singleNodeRoot = { children: [node], type: 'root' };
const transformedRoot = (await processorInstance.run(singleNodeRoot as MdastRoot)) as HastRoot;
const html = processorInstance.stringify(transformedRoot);
+ const safeHtml = allowHtml
+ ? (DOMPurify.sanitize(html, SAFE_HTML_CONFIG) as unknown as string)
+ : html;
- transformCache.set(hash, html);
+ transformCache.set(cacheKey, safeHtml);
- return { hash, html };
+ return { hash, html: safeHtml };
}
/**
@@ -300,7 +318,7 @@
if (prefixMarkdown.trim()) {
const normalizedPrefix = preprocessLaTeX(prefixMarkdown);
- const processorInstance = getMarkdownProcessor({ attachments, disableMath });
+ const processorInstance = getMarkdownProcessor({ allowHtml, attachments, disableMath });
const ast = processorInstance.parse(normalizedPrefix) as MdastRoot;
const mdastChildren = (ast as { children?: unknown[] }).children ?? [];
const nextBlocks: MarkdownBlock[] = [];
@@ -350,7 +368,7 @@
incompleteCodeBlock = null;
const normalized = preprocessLaTeX(markdown);
- const processorInstance = getMarkdownProcessor({ attachments, disableMath });
+ const processorInstance = getMarkdownProcessor({ allowHtml, attachments, disableMath });
const ast = processorInstance.parse(normalized) as MdastRoot;
const mdastChildren = (ast as { children?: unknown[] }).children ?? [];
const stableCount = Math.max(mdastChildren.length - 1, 0);
@@ -394,6 +412,10 @@
)) as HastRoot;
unstableHtml = processorInstance.stringify(transformedRoot);
+
+ if (allowHtml) {
+ unstableHtml = DOMPurify.sanitize(unstableHtml, SAFE_HTML_CONFIG) as unknown as string;
+ }
}
renderedBlocks = nextBlocks;
diff --git a/tools/ui/src/lib/components/app/content/MarkdownContent/markdown-processor.ts b/tools/ui/src/lib/components/app/content/MarkdownContent/markdown-processor.ts
index 57ded3e99..26cae5b5c 100644
--- a/tools/ui/src/lib/components/app/content/MarkdownContent/markdown-processor.ts
+++ b/tools/ui/src/lib/components/app/content/MarkdownContent/markdown-processor.ts
@@ -42,12 +42,15 @@ export interface MarkdownProcessor {
export interface MarkdownProcessorOptions {
attachments?: DatabaseMessageExtra[];
disableMath?: boolean;
+ /** Render raw HTML found in the markdown instead of escaping it. */
+ allowHtml?: boolean;
}
const sharedPipelines = new Map<string, MarkdownProcessor>();
-const attachmentPipelines = new WeakMap<object, MarkdownProcessor>();
+const attachmentPipelines = new WeakMap<object, Map<string, MarkdownProcessor>>();
function buildPipeline({
+ allowHtml = false,
attachments,
disableMath = false
}: MarkdownProcessorOptions): MarkdownProcessor {
@@ -58,11 +61,14 @@ function buildPipeline({
proc = proc.use(remarkMath); // Parse $inline$ and $$block$$ math
}
- proc = proc
- .use(remarkBreaks) // Convert line breaks to <br>
+ proc = proc.use(remarkBreaks); // Convert line breaks to <br>
+
+ if (!allowHtml) {
// Treat raw HTML as literal text with preserved indentation
- .use(remarkLiteralHtml)
- .use(remarkRehype); // Convert Markdown AST to rehype
+ proc = proc.use(remarkLiteralHtml);
+ }
+
+ proc = proc.use(remarkRehype, allowHtml ? { allowDangerousHtml: true } : undefined);
if (!disableMath) {
proc = proc.use(rehypeKatex); // Render math using KaTeX
@@ -91,17 +97,26 @@ function buildPipeline({
export function getMarkdownProcessor(options: MarkdownProcessorOptions): MarkdownProcessor {
if (options.attachments && options.attachments.length > 0) {
- let cached = attachmentPipelines.get(options.attachments);
+ let byOptions = attachmentPipelines.get(options.attachments);
+
+ if (!byOptions) {
+ byOptions = new Map<string, MarkdownProcessor>();
+ attachmentPipelines.set(options.attachments, byOptions);
+ }
+
+ const key = `${Boolean(options.disableMath)}:${Boolean(options.allowHtml)}`;
+
+ let cached = byOptions.get(key);
if (!cached) {
cached = buildPipeline(options);
- attachmentPipelines.set(options.attachments, cached);
+ byOptions.set(key, cached);
}
return cached;
}
- const key = String(Boolean(options.disableMath));
+ const key = `${Boolean(options.disableMath)}:${Boolean(options.allowHtml)}`;
let cached = sharedPipelines.get(key);
diff --git a/tools/ui/src/lib/components/app/content/MarkdownContent/safe-html-config.ts b/tools/ui/src/lib/components/app/content/MarkdownContent/safe-html-config.ts
new file mode 100644
index 000000000..d8dff07dd
--- /dev/null
+++ b/tools/ui/src/lib/components/app/content/MarkdownContent/safe-html-config.ts
@@ -0,0 +1,138 @@
+// Sanitization rules for raw HTML in model cards (MarkdownContent's
+// allowHtml mode). Kept as its own module so the adversarial test in
+// tests/client can import the exact shipped configuration.
+import DOMPurify from 'dompurify';
+
+// style is allowed for the KaTeX output, which lays its spans out with inline
+// styles; a third party card can use the same attribute to draw a fixed
+// overlay over the app, so positioned values are scrubbed
+DOMPurify.addHook('afterSanitizeAttributes', (node) => {
+ if (typeof (node as Element).getAttribute !== 'function') return;
+
+ const style = (node as Element).getAttribute('style');
+
+ if (!style || !/position\s*:\s*(fixed|absolute)/i.test(style)) return;
+
+ (node as Element).setAttribute(
+ 'style',
+ style.replace(/position\s*:\s*(fixed|absolute)\s*;?/gi, '')
+ );
+});
+
+/**
+ * Explicit DOMPurify allow list: the tags and attributes model cards may use
+ * plus the KaTeX output (spans with inline styles and MathML). Everything
+ * else - scripts, event handlers, forms, iframes, style tags - is stripped.
+ */
+export const SAFE_HTML_CONFIG = {
+ ALLOWED_ATTR: [
+ 'class',
+ 'style',
+ 'id',
+ 'title',
+ 'role',
+ 'aria-hidden',
+ 'aria-label',
+ 'aria-describedby',
+ 'aria-expanded',
+ 'href',
+ 'target',
+ 'rel',
+ 'src',
+ 'srcset',
+ 'alt',
+ 'width',
+ 'height',
+ 'align',
+ 'valign',
+ 'colspan',
+ 'rowspan',
+ 'controls',
+ 'type',
+ // MathML
+ 'xmlns',
+ 'display',
+ 'encoding',
+ 'mathvariant'
+ ],
+ ALLOWED_TAGS: [
+ // text and structure
+ 'a',
+ 'b',
+ 'i',
+ 'u',
+ 's',
+ 'em',
+ 'strong',
+ 'code',
+ 'pre',
+ 'kbd',
+ 'sub',
+ 'sup',
+ 'br',
+ 'hr',
+ 'p',
+ 'span',
+ 'div',
+ 'blockquote',
+ 'center',
+ 'ul',
+ 'ol',
+ 'li',
+ 'dl',
+ 'dt',
+ 'dd',
+ 'h1',
+ 'h2',
+ 'h3',
+ 'h4',
+ 'h5',
+ 'h6',
+ 'table',
+ 'thead',
+ 'tbody',
+ 'tr',
+ 'th',
+ 'td',
+ 'colgroup',
+ 'col',
+ 'caption',
+ // media
+ 'img',
+ 'picture',
+ 'source',
+ 'figure',
+ 'figcaption',
+ 'video',
+ 'audio',
+ // MathML, for the KaTeX output
+ 'math',
+ 'mrow',
+ 'mi',
+ 'mo',
+ 'mn',
+ 'ms',
+ 'mtext',
+ 'mfrac',
+ 'mroot',
+ 'msqrt',
+ 'msub',
+ 'msup',
+ 'msubsup',
+ 'munder',
+ 'mover',
+ 'mmultiscripts',
+ 'mtable',
+ 'mtr',
+ 'mtd',
+ 'mspace',
+ 'mglyph',
+ 'maligngroup',
+ 'malignmark',
+ 'mpadded',
+ 'mphantom',
+ 'mstyle',
+ 'semantics',
+ 'annotation'
+ ]
+};
diff --git a/tools/ui/src/lib/components/app/dialogs/DialogManageModels.svelte b/tools/ui/src/lib/components/app/dialogs/DialogManageModels.svelte
index 45b73dc72..fbe711c25 100644
--- a/tools/ui/src/lib/components/app/dialogs/DialogManageModels.svelte
+++ b/tools/ui/src/lib/components/app/dialogs/DialogManageModels.svelte
@@ -1,7 +1,18 @@
<script lang="ts">
+ import { ArrowLeft } from '@lucide/svelte';
+ import ModelsDiscover from '$lib/components/app/models/discover/ModelsDiscover.svelte';
import ModelsManager from '$lib/components/app/models/ModelsManager/ModelsManager.svelte';
+ import { Button } from '$lib/components/ui/button';
import * as Dialog from '$lib/components/ui/dialog';
- import { MODEL_ICON } from '$lib/constants';
+ import {
+ DISCOVER_ICON,
+ MODEL_ICON,
+ MODELS_DIALOG_VIEW_FADE_MS,
+ SETTINGS_KEYS
+ } from '$lib/constants';
+ import { ModelsDialogView } from '$lib/enums';
+ import { modelsStore, settingsStore, uiStore } from '$lib/stores';
+ import { untrack } from 'svelte';
interface Props {
open?: boolean;
@@ -10,6 +21,85 @@
let { onOpenChange, open = $bindable(false) }: Props = $props();
+ // what the user asked for, and what is actually rendered: a view change fades the
+ // current one out, swaps, then fades the next one in
+ let view = $state<ModelsDialogView>(ModelsDialogView.MANAGE);
+ let shownView = $state<ModelsDialogView>(ModelsDialogView.MANAGE);
+ let isSwapping = $state(false);
+ /** Set while an entry opens the dialog on an explicit view, so the open reset keeps hands off. */
+ let viewRequested = false;
+
+ // the discover view exists only while its setting is on
+ let discoverEnabled = $derived(
+ Boolean(settingsStore.getConfig(SETTINGS_KEYS.ENABLE_DISCOVER_MODELS))
+ );
+
+ $effect(() => {
+ const next = view;
+
+ if (next === shownView) return;
+
+ // nothing to fade while the dialog is closed, so the view is set before it opens
+ if (!open) {
+ untrack(() => (shownView = next));
+
+ return;
+ }
+
+ untrack(() => (isSwapping = true));
+
+ const timer = setTimeout(() => {
+ untrack(() => {
+ shownView = next;
+ isSwapping = false;
+ });
+ }, MODELS_DIALOG_VIEW_FADE_MS);
+
+ return () => clearTimeout(timer);
+ });
+
+ let title = $derived(view === ModelsDialogView.DISCOVER ? 'Discover' : 'Models');
+
+ // the sidebar's Discover entry opens this dialog on its Discover view
+ $effect(() => {
+ if (!uiStore.discoverModelsOpen) return;
+
+ untrack(() => {
+ uiStore.discoverModelsOpen = false;
+ view = discoverEnabled ? ModelsDialogView.DISCOVER : ModelsDialogView.MANAGE;
+
+ // the dialog may already be open, so no open transition will follow
+ if (open) return;
+
+ viewRequested = true;
+ handleOpenChange(true);
+ });
+ });
+
+ // opening always lands on the manager, unless an entry requested the discover view
+ $effect(() => {
+ if (!open) return;
+
+ untrack(() => {
+ if (viewRequested) {
+ viewRequested = false;
+ } else {
+ view = ModelsDialogView.MANAGE;
+ }
+
+ // sync point: the table opens on cached state and corrects itself when
+ // the list fetch lands; a no-op outside router mode
+ void modelsStore.fetchRouterModels();
+ });
+ });
+
+ // the discover view disappears with its setting; fall back to the manager
+ $effect(() => {
+ if (!discoverEnabled && view === ModelsDialogView.DISCOVER) {
+ untrack(() => (view = ModelsDialogView.MANAGE));
+ }
+ });
+
function handleOpenChange(value: boolean) {
open = value;
onOpenChange?.(value);
@@ -21,11 +111,49 @@
class="max-md:h-[100dvh]! max-md:w-screen! max-md:max-w-none! max-md:rounded-none! max-md:border-0! md:h-[calc(100vh-4rem)]! md:max-h-240! md:w-[calc(100vw-4rem)]! md:max-w-380! flex flex-col p-4 pb-0"
onOpenAutoFocus={(event) => event.preventDefault()}
>
- <Dialog.Header class="flex flex-row items-center pr-8">
- <Dialog.Title class="flex min-h-7 items-center gap-2">
- <MODEL_ICON class="h-5 w-5 shrink-0" />
+ <Dialog.Header class="relative flex flex-row items-center justify-between p-2 pr-8">
+ <!--
+ The arrow is out of the flow and the title slides, so entering a sub-view
+ moves the title with a transform instead of reflowing the header. min-h
+ keeps the header's height fixed either way.
+ -->
+ <Button
+ aria-label="Back to models"
+ class="absolute top-1/2 left-1 h-7 w-7 -translate-y-1/2 transition-[opacity,visibility] duration-200 ease-[cubic-bezier(0.23,1,0.32,1)] {view ===
+ ModelsDialogView.MANAGE
+ ? 'invisible opacity-0'
+ : 'visible opacity-100'}"
+ onclick={() => (view = ModelsDialogView.MANAGE)}
+ size="icon"
+ variant="ghost"
+ >
+ <ArrowLeft class="h-4 w-4" />
+ </Button>
- <span>Models</span>
+ <Dialog.Title
+ class="flex min-h-7 items-center gap-2 transition-transform duration-200 ease-[cubic-bezier(0.23,1,0.32,1)] {view ===
+ ModelsDialogView.MANAGE
+ ? 'translate-x-0'
+ : 'translate-x-8'}"
+ >
+ <!-- the marks cross-fade in one grid cell, so the title never shifts -->
+ <span class="grid h-5 w-5 shrink-0 place-items-center">
+ <MODEL_ICON
+ class="col-start-1 row-start-1 h-5 w-5 transition-opacity duration-150 {view ===
+ ModelsDialogView.MANAGE
+ ? 'opacity-100'
+ : 'opacity-0'}"
+ />
+
+ <DISCOVER_ICON
+ class="col-start-1 row-start-1 h-5 w-5 transition-opacity duration-150 {view ===
+ ModelsDialogView.DISCOVER
+ ? 'opacity-100'
+ : 'opacity-0'}"
+ />
+ </span>
+
+ <span>{title}</span>
</Dialog.Title>
<Dialog.Description class="sr-only">
@@ -33,8 +161,47 @@
</Dialog.Description>
</Dialog.Header>
- <div class="min-h-0 flex-1">
- <ModelsManager class="h-full" />
+ <div class="dialog-view min-h-0 flex-1" data-visible={!isSwapping}>
+ {#if shownView === ModelsDialogView.MANAGE}
+ <ModelsManager class="h-full">
+ {#snippet toolbarEnd()}
+ {#if discoverEnabled}
+ <Button
+ class="gap-1.5"
+ onclick={() => (view = ModelsDialogView.DISCOVER)}
+ size="sm"
+ variant="tertiary"
+ >
+ <DISCOVER_ICON class="h-3.5 w-3.5" />
+
+ Discover Models
+ </Button>
+ {/if}
+ {/snippet}
+ </ModelsManager>
+ {:else if shownView === ModelsDialogView.DISCOVER}
+ <div class="grid h-full overflow-hidden" style="grid-template-columns: auto 1fr;">
+ <ModelsDiscover />
+ </div>
+ {/if}
</div>
</Dialog.Content>
</Dialog.Root>
+
+<style>
+ /* a view change reads as one surface swapping, not as content teleporting */
+ .dialog-view {
+ opacity: 0;
+ transition: opacity 150ms cubic-bezier(0.23, 1, 0.32, 1);
+ }
+
+ .dialog-view[data-visible='true'] {
+ opacity: 1;
+ }
+
+ @media (prefers-reduced-motion: reduce) {
+ .dialog-view {
+ transition-duration: 100ms;
+ }
+ }
+</style>
diff --git a/tools/ui/src/lib/components/app/models/ModelDraftSidecars.svelte b/tools/ui/src/lib/components/app/models/ModelDraftSidecars.svelte
index 9b655f843..4e7b07156 100644
--- a/tools/ui/src/lib/components/app/models/ModelDraftSidecars.svelte
+++ b/tools/ui/src/lib/components/app/models/ModelDraftSidecars.svelte
@@ -31,8 +31,6 @@
</script>
{#each badges as badge (badge.kind + (badge.quant ?? ''))}
- <span aria-hidden="true" class="text-muted-foreground">+</span>
-
<span
class={MODEL_VARIANT_BADGE_CLASS}
title="{badge.kind.toUpperCase()} draft model{badge.quant ? ` at ${badge.quant}` : ''}"
diff --git a/tools/ui/src/lib/components/app/models/ModelDraftSubline.svelte b/tools/ui/src/lib/components/app/models/ModelDraftSubline.svelte
new file mode 100644
index 000000000..101215d11
--- /dev/null
+++ b/tools/ui/src/lib/components/app/models/ModelDraftSubline.svelte
@@ -0,0 +1,53 @@
+<script lang="ts">
+ import { Check } from '@lucide/svelte';
+ import { MODEL_BADGE_CLASS, MODEL_VARIANT_BADGE_CLASS } from '$lib/constants';
+ import { ModelRowDownloadState } from '$lib/enums';
+
+ interface Props {
+ /**
+ * Left inset of the tree connector, tuned per surface so the elbow hangs
+ * under the main line's avatar and "Draft model" aligns with its id.
+ */
+ connectorClass?: string;
+ /** Draft sidecar kind, e.g. dflash. */
+ kind: string;
+ /** Quant of the draft, when its tag carries one. */
+ quant?: string | null;
+ /** Live state of the draft download. */
+ state: ModelRowDownloadState;
+ /** Rounded download percent, when the feed reports progress. */
+ percent?: number | null;
+ }
+
+ let { connectorClass = 'ml-6', kind, percent = null, quant = null, state }: Props = $props();
+</script>
+
+<!-- the compact draft line lives inside its main item, under the id line; the
+ main model's controls pause, resume and cancel its quant-matched drafts, so
+ the line only reports its state -->
+<div class="flex items-center gap-2 py-1 pr-2 text-sm">
+ <span
+ aria-hidden="true"
+ class="h-2.5 w-3.5 shrink-0 self-center rounded-bl-sm border-b border-l border-border/70 {connectorClass}"
+ ></span>
+
+ <span class="shrink-0 text-xs text-muted-foreground">Draft model</span>
+
+ <span class={MODEL_VARIANT_BADGE_CLASS}>{kind}</span>
+
+ {#if quant}
+ <span class={MODEL_BADGE_CLASS}>{quant}</span>
+ {/if}
+
+ {#if state === ModelRowDownloadState.DOWNLOADED}
+ <Check class="ml-auto h-4 w-4 shrink-0 text-green-500" />
+ {:else}
+ <span class="ml-auto flex shrink-0 items-center">
+ {#if percent !== null}
+ <span class="font-mono text-xs tabular-nums text-muted-foreground">{percent}%</span>
+ {:else if state === ModelRowDownloadState.PAUSED}
+ <span class="text-xs text-muted-foreground">Paused</span>
+ {/if}
+ </span>
+ {/if}
+</div>
diff --git a/tools/ui/src/lib/components/app/models/ModelId.svelte b/tools/ui/src/lib/components/app/models/ModelId.svelte
index 26d72f662..9a3464ab8 100644
--- a/tools/ui/src/lib/components/app/models/ModelId.svelte
+++ b/tools/ui/src/lib/components/app/models/ModelId.svelte
@@ -5,6 +5,7 @@
import { TruncatedText } from '$lib/components/app';
import * as Tooltip from '$lib/components/ui/tooltip';
import { MODEL_BADGE_CLASS, MODEL_VARIANT_BADGE_CLASS } from '$lib/constants';
+ import type { ModelDraftSidecar } from '$lib/enums';
import { HuggingFaceService } from '$lib/services';
import { ModelsService } from '$lib/services/models.service';
import { settingsStore } from '$lib/stores';
@@ -39,6 +40,8 @@
sizeRange?: { min: number; max: number } | null;
/** Draft sidecars available for the model, badged with their own quant. */
draftSidecars?: ModelSidecarBadge[];
+ /** Draft sidecar kinds the repo offers, one badge per kind and no quant. */
+ draftKinds?: ModelDraftSidecar[];
/** Give the id a row of its own, with its badges and icons together under it. */
stackId?: boolean;
/** Allow badges to wrap onto new lines instead of truncating. */
@@ -50,6 +53,7 @@
aliases,
class: className = '',
contextLength,
+ draftKinds = [],
draftSidecars = [],
hideCapabilities = false,
hideModalities = false,
@@ -93,12 +97,17 @@
let uniqueAliases = $derived([...new Set(aliases ?? [])]);
let uniqueTags = $derived([...new Set([...(parsed.tags ?? []), ...(tags ?? [])])]);
let uniqueDraftSidecars = $derived(draftSidecars.filter((badge) => !isAuxSidecar(badge.kind)));
+ // the model id can already name one of the kinds
+ let uniqueDraftKinds = $derived([
+ ...new Set(draftKinds.filter((kind) => kind !== parsed.sidecar))
+ ]);
let primaryAlias = $derived(uniqueAliases.length === 1 ? uniqueAliases[0] : null);
let displayName = $derived(primaryAlias ?? parsed.modelName ?? modelId);
let hasBadges = $derived(
parsed.sidecar ||
+ uniqueDraftKinds.length > 0 ||
uniqueDraftSidecars.length > 0 ||
(parsed.params && !hideParameters) ||
(parsed.quantization && !resolvedHideQuantization) ||
@@ -144,6 +153,12 @@
</span>
{/if}
+ {#each uniqueDraftKinds as kind (kind)}
+ <span class={variantBadgeClass} title="{kind.toUpperCase()} draft model">
+ {kind}
+ </span>
+ {/each}
+
<ModelDraftSidecars {draftSidecars} />
{#if primaryAlias}
diff --git a/tools/ui/src/lib/components/app/models/ModelsManager/ModelsManager.svelte b/tools/ui/src/lib/components/app/models/ModelsManager/ModelsManager.svelte
index 36fd3e22a..9853bafe5 100644
--- a/tools/ui/src/lib/components/app/models/ModelsManager/ModelsManager.svelte
+++ b/tools/ui/src/lib/components/app/models/ModelsManager/ModelsManager.svelte
@@ -32,9 +32,22 @@
let capabilityFilter = $state<ModelCapability[]>([]);
let selectedId = $state<string | null>(null);
+ // tracked downloads get their own section, listed like any other model
+ let downloads = $derived(
+ downloadGroups(modelsStore.status.getDownloadEntries(), modelsStore.models, (repo) =>
+ modelsStore.status.getDraftSidecars(repo)
+ )
+ );
+
let allModels = $derived(modelsStore.models);
- let selected = $derived(allModels.find((option) => option.id === selectedId) ?? null);
+ // the download groups stand in for rows the listing does not carry yet, so a
+ // click on them still finds its option and opens the pane
+ let selected = $derived(
+ allModels.find((option) => option.id === selectedId) ??
+ downloads.find((group) => group.base.id === selectedId)?.base ??
+ null
+ );
// The pane is laid out before it is ever opened, so the first open only slides a
// finished panel in. It renders the selection, else the model it last showed.
let lastPicked = $state<ModelOption | null>(null);
@@ -42,7 +55,11 @@
let shownId = $state<string | null>(null);
let isSwapping = $state(false);
let fade = $state<'open' | 'swap'>('open');
- let shownOption = $derived(allModels.find((option) => option.id === shownId) ?? null);
+ let shownOption = $derived(
+ allModels.find((option) => option.id === shownId) ??
+ downloads.find((group) => group.base.id === shownId)?.base ??
+ null
+ );
$effect(() => {
const id = selectedId;
@@ -144,10 +161,6 @@
rank.size === 0 ? list : [...list].sort((a, b) => rankOf(a) - rankOf(b));
// one entry per repo, so a model with several quants takes a single table row
let entries = $derived(byRecency(groupModelQuants(matching)));
- // tracked downloads get their own section, listed like any other model
- let downloads = $derived(
- downloadGroups(modelsStore.status.getDownloadEntries(), modelsStore.models)
- );
let groups = $derived.by(() => {
// a loaded quant is a model of its own: its repo keeps the quants left behind
const isLoaded = (option: ModelOption) => modelsStore.isModelLoaded(option.model);
@@ -216,13 +229,16 @@
return ordered;
});
- // a caller can ask for one model to be revealed, the download rows do
+ // a caller can ask for one model to be revealed, the download rows do; a
+ // tracked download is not in the listing yet, so its own rows are searched too
$effect(() => {
const focus = uiStore.manageModelFocus;
if (!focus) return;
- const option = allModels.find((model) => model.id === focus || model.model === focus);
+ const matches = (candidate: ModelOption) => candidate.id === focus || candidate.model === focus;
+ const option =
+ allModels.find(matches) ?? downloads.find((group) => matches(group.base))?.base ?? null;
if (option) selectedId = option.id;
diff --git a/tools/ui/src/lib/components/app/models/ModelsManager/ModelsManagerDownloadControl.svelte b/tools/ui/src/lib/components/app/models/ModelsManager/ModelsManagerDownloadControl.svelte
index 0e81500f2..5e77caed0 100644
--- a/tools/ui/src/lib/components/app/models/ModelsManager/ModelsManagerDownloadControl.svelte
+++ b/tools/ui/src/lib/components/app/models/ModelsManager/ModelsManagerDownloadControl.svelte
@@ -38,10 +38,7 @@
class="h-5 w-5 hover:text-foreground"
icon={isPaused ? MODEL_DOWNLOAD_ICONS.resume : MODEL_DOWNLOAD_ICONS.pause}
iconSize="h-4 w-4"
- onclick={() =>
- void (isPaused
- ? modelsStore.status.downloadModel(option.model)
- : modelsStore.status.pauseDownload(option.model))}
+ onclick={() => void modelsStore.status.toggleDownload(option.model)}
stopPropagationOnClick
tooltip={isPaused ? 'Resume downloading' : 'Pause downloading'}
tooltipAsTitle
diff --git a/tools/ui/src/lib/components/app/models/ModelsManager/ModelsManagerModelConfiguration/ModelsManagerModelConfiguration.svelte b/tools/ui/src/lib/components/app/models/ModelsManager/ModelsManagerModelConfiguration/ModelsManagerModelConfiguration.svelte
index 8a109a8db..40a7565ed 100644
--- a/tools/ui/src/lib/components/app/models/ModelsManager/ModelsManagerModelConfiguration/ModelsManagerModelConfiguration.svelte
+++ b/tools/ui/src/lib/components/app/models/ModelsManager/ModelsManagerModelConfiguration/ModelsManagerModelConfiguration.svelte
@@ -1,4 +1,6 @@
<script lang="ts">
+ import ModelDownloadProgressBar from '../../ModelDownloadProgressBar.svelte';
+ import { downloadProgressFor } from '../utils';
import ModelsManagerModelConfigurationActions from './ModelsManagerModelConfigurationActions.svelte';
import ModelsManagerModelConfigurationHeader from './ModelsManagerModelConfigurationHeader.svelte';
import ModelsManagerModelConfigurationInformation from './ModelsManagerModelConfigurationInformation.svelte';
@@ -7,7 +9,7 @@
import { Button } from '$lib/components/ui/button';
import { ModelDownloadConfirmAction, ServerModelStatus } from '$lib/enums';
import { HuggingFaceService } from '$lib/services';
- import { deviceStore, modelsStore } from '$lib/stores';
+ import { deviceStore, modelsStore, uiStore } from '$lib/stores';
import type { HfModelDetailInfo } from '$lib/types/huggingface';
import type { ModelOption } from '$lib/types/models';
import { repoOf } from '$lib/utils';
@@ -34,6 +36,23 @@
let status = $derived(modelsStore.getModelStatus(option.model));
let isLoaded = $derived(modelsStore.isModelRunning(option.model));
+ // an in-flight download has no configuration: the pane shows its progress and
+ // the download actions, everything a loaded or downloaded model offers waits
+ let isDownload = $derived(
+ modelsStore.status.isDownloadInProgress(option.model) ||
+ modelsStore.status.isDownloadPaused(option.model)
+ );
+ let isDownloadPaused = $derived(modelsStore.status.isDownloadPaused(option.model));
+ let downloadProgress = $derived(downloadProgressFor(option.model));
+ let downloadPercent = $derived(
+ downloadProgress && downloadProgress.totalBytes > 0
+ ? Math.round((downloadProgress.downloadedBytes / downloadProgress.totalBytes) * 100)
+ : null
+ );
+
+ // the same cancellation the table's rows confirm, owned by this pane
+ let cancelDownloadOpen = $state(false);
+
// The server reports the full metadata only once a model is loaded, which loads it.
// With the Hub enabled, it fills those gaps instead.
let hubDetails = $state<HfModelDetailInfo | null>(null);
@@ -63,8 +82,15 @@
<div class="flex h-full min-h-0 flex-col">
<ModelsManagerModelConfigurationHeader
+ {isDownload}
+ {isDownloadPaused}
{isLoaded}
{onClose}
+ onRequestCancelDownload={() => (cancelDownloadOpen = true)}
+ onSeeInDiscover={isDownload
+ ? () => uiStore.openModelsDiscover(repoOf(option.model))
+ : undefined}
+ onToggleDownload={() => void modelsStore.status.toggleDownload(option.model)}
{onToggleLoad}
{onUseInChat}
{onUseInNewChat}
@@ -73,27 +99,63 @@
/>
<div class="min-h-0 flex-1 overflow-y-auto py-4 pl-4 max-md:pr-4">
- <ModelsManagerModelConfigurationInformation hub={hubDetails} {option} {serverProps} />
-
- <div class="mt-4 border-t border-border/30 pt-3 pb-2">
- <Button
- class="w-full justify-start gap-2 text-destructive hover:bg-destructive/10 hover:text-destructive dark:hover:bg-destructive/20"
- onclick={() => (deleteOpen = true)}
- size="sm"
- variant="ghost"
- >
- <Trash2 class="h-4 w-4" />
- Delete this model from disk
- </Button>
- </div>
+ {#if isDownload}
+ <!-- nothing is on disk yet, so the pane keeps to the download itself -->
+ <div class="space-y-2 py-1">
+ <div class="flex items-baseline justify-between gap-3 text-sm">
+ <span class="text-muted-foreground">
+ {isDownloadPaused ? 'Paused at' : 'Downloaded'}
+ </span>
+
+ <span class="font-mono text-xs tabular-nums text-muted-foreground">
+ {#if downloadPercent !== null}
+ {HuggingFaceService.formatFileSize(downloadProgress?.downloadedBytes ?? 0)}
+ /
+ {HuggingFaceService.formatFileSize(downloadProgress?.totalBytes ?? 0)}
+ ({downloadPercent}%)
+ {:else}
+ Waiting for progress...
+ {/if}
+ </span>
+ </div>
+
+ {#if downloadProgress && downloadProgress.totalBytes > 0}
+ <ModelDownloadProgressBar
+ downloadedBytes={downloadProgress.downloadedBytes}
+ totalBytes={downloadProgress.totalBytes}
+ />
+ {/if}
+ </div>
+ {:else}
+ <ModelsManagerModelConfigurationInformation hub={hubDetails} {option} {serverProps} />
+
+ <div class="mt-4 border-t border-border/30 pt-3 pb-2">
+ <Button
+ class="w-full justify-start gap-2 text-destructive hover:bg-destructive/10 hover:text-destructive dark:hover:bg-destructive/20"
+ onclick={() => (deleteOpen = true)}
+ size="sm"
+ variant="ghost"
+ >
+ <Trash2 class="h-4 w-4" />
+ Delete this model from disk
+ </Button>
+ </div>
+ {/if}
</div>
<!-- a phone keeps the actions at the bottom of its screen, in reach of the thumb -->
{#if deviceStore.isMobile}
<div class="shrink-0 border-t border-border/40 px-4 py-3">
<ModelsManagerModelConfigurationActions
+ {isDownload}
+ {isDownloadPaused}
{isLoaded}
isLoading={status === ServerModelStatus.LOADING}
+ onRequestCancelDownload={() => (cancelDownloadOpen = true)}
+ onSeeInDiscover={isDownload
+ ? () => uiStore.openModelsDiscover(repoOf(option.model))
+ : undefined}
+ onToggleDownload={() => void modelsStore.status.toggleDownload(option.model)}
{onToggleLoad}
{onUseInChat}
{onUseInNewChat}
@@ -107,4 +169,12 @@
open={deleteOpen}
repoWithTag={option.model}
/>
+
+ <DialogConfirmDownload
+ action={ModelDownloadConfirmAction.CANCEL}
+ onClose={() => (cancelDownloadOpen = false)}
+ onConfirm={() => void modelsStore.status.cancelDownload(option.model)}
+ open={cancelDownloadOpen}
+ repoWithTag={option.model}
+ />
</div>
diff --git a/tools/ui/src/lib/components/app/models/ModelsManager/ModelsManagerModelConfiguration/ModelsManagerModelConfigurationActions.svelte b/tools/ui/src/lib/components/app/models/ModelsManager/ModelsManagerModelConfiguration/ModelsManagerModelConfigurationActions.svelte
index b7fc10997..affab3421 100644
--- a/tools/ui/src/lib/components/app/models/ModelsManager/ModelsManagerModelConfiguration/ModelsManagerModelConfigurationActions.svelte
+++ b/tools/ui/src/lib/components/app/models/ModelsManager/ModelsManagerModelConfiguration/ModelsManagerModelConfigurationActions.svelte
@@ -1,13 +1,31 @@
<script lang="ts">
- import { Eject, Loader2, MessageSquare, Power, SquarePen } from '@lucide/svelte';
+ import {
+ Compass,
+ Eject,
+ Loader2,
+ MessageSquare,
+ Pause,
+ Play,
+ Power,
+ SquarePen,
+ Trash2
+ } from '@lucide/svelte';
import { Button } from '$lib/components/ui/button';
- import { conversationsStore, deviceStore } from '$lib/stores';
+ import { SETTINGS_KEYS } from '$lib/constants';
+ import { conversationsStore, deviceStore, settingsStore } from '$lib/stores';
interface Props {
/** A provider that cannot load or unload has no button to offer. */
canToggleLoad?: boolean;
+ isDownload?: boolean;
+ isDownloadPaused?: boolean;
isLoaded: boolean;
isLoading: boolean;
+ /** Opens the manager dialog on the Discover view, focused on this model's repo. */
+ onSeeInDiscover?: () => void;
+ /** Confirm-cancel dialog on the pane, so nothing deletes without a click. */
+ onRequestCancelDownload?: () => void;
+ onToggleDownload?: () => void;
onToggleLoad: () => void;
onUseInChat: () => void;
onUseInNewChat: () => void;
@@ -17,14 +35,24 @@
let {
canToggleLoad = true,
+ isDownload = false,
+ isDownloadPaused = false,
isLoaded,
isLoading,
+ onRequestCancelDownload,
+ onSeeInDiscover,
+ onToggleDownload,
onToggleLoad,
onUseInChat,
onUseInNewChat,
size = 'sm'
}: Props = $props();
+ // the discover view exists only while its setting is on
+ let discoverEnabled = $derived(
+ Boolean(settingsStore.getConfig(SETTINGS_KEYS.ENABLE_DISCOVER_MODELS))
+ );
+
// a chat with anything in it takes the model, a new chat is not what its user asked for
let hasChat = $derived(
conversationsStore.activeConversation !== null ||
@@ -34,52 +62,91 @@
</script>
<div class="my-3 flex flex-col gap-2">
- {#if isMobile}
- <!-- a phone picks its model in the manager, so pointing this chat at it leads -->
- <Button class="w-full gap-1.5" onclick={hasChat ? onUseInChat : onUseInNewChat} {size}>
- {#if hasChat}
- <MessageSquare class="h-3.5 w-3.5" />
+ {#if isDownload}
+ <!-- an in-flight download cannot load or chat: it pauses, resumes, cancels -->
+ <div class="flex gap-3">
+ <Button class="flex-1 gap-1.5" onclick={onToggleDownload} {size} variant="secondary">
+ {#if isDownloadPaused}
+ <Play class="h-3.5 w-3.5" />
- Use in this chat
- {:else}
- <SquarePen class="h-3.5 w-3.5" />
+ Resume downloading
+ {:else}
+ <Pause class="h-3.5 w-3.5" />
- Start a new chat
- {/if}
- </Button>
- {/if}
+ Pause downloading
+ {/if}
+ </Button>
- <div class="flex gap-3 mt-2">
- {#if !isMobile}
- <Button class="flex-1 gap-1.5" onclick={onUseInNewChat} {size}>
- <SquarePen class="h-3.5 w-3.5" />
+ <Button class="flex-1 gap-1.5" onclick={onRequestCancelDownload} {size} variant="secondary">
+ <Trash2 class="h-3.5 w-3.5" />
- Start a new chat
+ Cancel download
+ </Button>
+ </div>
+
+ {#if onSeeInDiscover && discoverEnabled}
+ <Button class="gap-1.5" onclick={onSeeInDiscover} {size} variant="tertiary">
+ <Compass class="h-3.5 w-3.5" />
+
+ See in Discover
</Button>
{/if}
+ {:else}
+ {#if isMobile}
+ <!-- a phone picks its model in the manager, so pointing this chat at it leads -->
+ <Button class="w-full gap-1.5" onclick={hasChat ? onUseInChat : onUseInNewChat} {size}>
+ {#if hasChat}
+ <MessageSquare class="h-3.5 w-3.5" />
- {#if canToggleLoad}
- <Button
- class="flex-1 gap-1.5"
- disabled={isLoading}
- onclick={onToggleLoad}
- {size}
- variant="secondary"
- >
- {#if isLoading}
- <Loader2 class="h-3.5 w-3.5 animate-spin" />
-
- Loading...
- {:else if isLoaded}
- <Eject class="h-3.5 w-3.5" />
-
- Unload model
+ Use in this chat
{:else}
- <Power class="h-3.5 w-3.5" />
+ <SquarePen class="h-3.5 w-3.5" />
- Load model
+ Start a new chat
{/if}
</Button>
{/if}
- </div>
+
+ <div class="flex gap-3 mt-2">
+ {#if !isMobile}
+ <Button class="flex-1 gap-1.5" onclick={onUseInNewChat} {size}>
+ <SquarePen class="h-3.5 w-3.5" />
+
+ Start a new chat
+ </Button>
+ {/if}
+
+ {#if canToggleLoad}
+ <Button
+ class="flex-1 gap-1.5"
+ disabled={isLoading}
+ onclick={onToggleLoad}
+ {size}
+ variant="secondary"
+ >
+ {#if isLoading}
+ <Loader2 class="h-3.5 w-3.5 animate-spin" />
+
+ Loading...
+ {:else if isLoaded}
+ <Eject class="h-3.5 w-3.5" />
+
+ Unload model
+ {:else}
+ <Power class="h-3.5 w-3.5" />
+
+ Load model
+ {/if}
+ </Button>
+ {/if}
+
+ {#if onSeeInDiscover && discoverEnabled}
+ <Button class="flex-1 gap-1.5" onclick={onSeeInDiscover} {size} variant="tertiary">
+ <Compass class="h-3.5 w-3.5" />
+
+ See in Discover
+ </Button>
+ {/if}
+ </div>
+ {/if}
</div>
diff --git a/tools/ui/src/lib/components/app/models/ModelsManager/ModelsManagerModelConfiguration/ModelsManagerModelConfigurationHeader.svelte b/tools/ui/src/lib/components/app/models/ModelsManager/ModelsManagerModelConfiguration/ModelsManagerModelConfigurationHeader.svelte
index 40f5d0d54..d2cc2d5b8 100644
--- a/tools/ui/src/lib/components/app/models/ModelsManager/ModelsManagerModelConfiguration/ModelsManagerModelConfigurationHeader.svelte
+++ b/tools/ui/src/lib/components/app/models/ModelsManager/ModelsManagerModelConfiguration/ModelsManagerModelConfigurationHeader.svelte
@@ -8,8 +8,14 @@
import type { ModelOption } from '$lib/types/models';
interface Props {
+ isDownload?: boolean;
+ isDownloadPaused?: boolean;
isLoaded: boolean;
onClose: () => void;
+ /** Opens the manager dialog on the Discover view, focused on this model's repo. */
+ onSeeInDiscover?: () => void;
+ onRequestCancelDownload?: () => void;
+ onToggleDownload?: () => void;
onToggleLoad: () => void;
onUseInChat: () => void;
onUseInNewChat: () => void;
@@ -18,13 +24,27 @@
status: ServerModelStatus | null;
}
- let { isLoaded, onClose, onToggleLoad, onUseInChat, onUseInNewChat, option, status }: Props =
- $props();
+ let {
+ isDownload = false,
+ isDownloadPaused = false,
+ isLoaded,
+ onClose,
+ onRequestCancelDownload,
+ onSeeInDiscover,
+ onToggleDownload,
+ onToggleLoad,
+ onUseInChat,
+ onUseInNewChat,
+ option,
+ status
+ }: Props = $props();
let isMobile = $derived(deviceStore.isMobile);
let isLoading = $derived(status === ServerModelStatus.LOADING);
let statusLabel = $derived.by(() => {
+ if (isDownload) return isDownloadPaused ? 'Paused' : 'Downloading';
+
if (status === ServerModelStatus.LOADING) return 'Loading';
if (status === ServerModelStatus.FAILED) return 'Failed to load';
@@ -37,6 +57,8 @@
});
let statusDot = $derived.by(() => {
+ if (isDownload) return isDownloadPaused ? 'bg-yellow-500' : 'bg-blue-500 animate-pulse';
+
if (status === ServerModelStatus.FAILED) return 'bg-red-500';
if (status === ServerModelStatus.LOADING) return 'bg-muted-foreground/50 animate-pulse';
@@ -100,8 +122,13 @@
<!-- a phone floats the actions at the bottom of the pane, where the thumb is -->
{#if !isMobile}
<ModelsManagerModelConfigurationActions
+ {isDownload}
+ {isDownloadPaused}
{isLoaded}
{isLoading}
+ {onRequestCancelDownload}
+ {onSeeInDiscover}
+ {onToggleDownload}
{onToggleLoad}
{onUseInChat}
{onUseInNewChat}
diff --git a/tools/ui/src/lib/components/app/models/ModelsManager/ModelsManagerModelRow.svelte b/tools/ui/src/lib/components/app/models/ModelsManager/ModelsManagerModelRow.svelte
index 2ded1926c..204d746d8 100644
--- a/tools/ui/src/lib/components/app/models/ModelsManager/ModelsManagerModelRow.svelte
+++ b/tools/ui/src/lib/components/app/models/ModelsManager/ModelsManagerModelRow.svelte
@@ -3,6 +3,7 @@
import ModelCapabilities from '../ModelCapabilities.svelte';
import ModelContext from '../ModelContext.svelte';
import ModelDownloadProgressBar from '../ModelDownloadProgressBar.svelte';
+ import ModelDraftSubline from '../ModelDraftSubline.svelte';
import ModelId from '../ModelId.svelte';
import ModelsManagerStatusCell from './ModelsManagerStatusCell.svelte';
import { modelRowActions } from './row-actions';
@@ -12,7 +13,7 @@
import { MODEL_ROW_GRID_CLASS, MODEL_ROW_TRAILING_CELL_CLASS } from '$lib/constants';
import { ModelRowDownloadState } from '$lib/enums';
import { deviceStore, modelsStore } from '$lib/stores';
- import type { ModelOption } from '$lib/types/models';
+ import type { ModelOption, ModelSidecarBadge } from '$lib/types/models';
interface Props {
isFavorite: (option: ModelOption) => boolean;
@@ -20,11 +21,21 @@
onDelete: (option: ModelOption) => void;
onSelect: (option: ModelOption) => void;
selected: boolean;
+ /** Draft sidecars already on disk, when the row stands for a download. */
+ sidecarBadges?: ModelSidecarBadge[];
/** Left padding in px, from the nesting depth. */
indent?: number;
}
- let { indent = 0, isFavorite, onDelete, onSelect, option, selected }: Props = $props();
+ let {
+ indent = 0,
+ isFavorite,
+ onDelete,
+ onSelect,
+ option,
+ selected,
+ sidecarBadges
+ }: Props = $props();
let favorite = $derived(isFavorite(option));
let isHidden = $derived(modelsStore.isHidden(option.id));
@@ -38,6 +49,10 @@
? ModelRowDownloadState.DOWNLOADING
: null
);
+
+ function activate(): void {
+ onSelect(option);
+ }
</script>
<div
@@ -48,41 +63,58 @@
selected ? 'bg-accent text-accent-foreground' : 'hover:bg-muted/40'
]}
>
- <!-- the row holds a load control and an actions menu, so only the model itself is
+ <div class="flex min-w-0 flex-col">
+ <!-- the row holds a load control and an actions menu, so only the model itself is
the button: a button nested in a role="button" row is invalid -->
- <button
- aria-pressed={selected}
- class="flex min-w-0 cursor-pointer items-center gap-3 text-left max-md:gap-2 max-md:pr-9"
- onclick={() => onSelect(option)}
- style="padding-left: {indent}px"
- type="button"
- >
- <ModelAvatar {option} size="size-7 md:size-9" />
+ <button
+ aria-pressed={download ? undefined : selected}
+ class="flex min-w-0 cursor-pointer items-center gap-3 text-left max-md:gap-2 max-md:pr-9"
+ onclick={activate}
+ style="padding-left: {indent}px"
+ type="button"
+ >
+ <ModelAvatar {option} size="size-7 md:size-9" />
- <span class="flex min-w-0 items-center gap-1.25">
- <ModelId
- aliases={option.aliases}
- class="min-w-0 flex-1 max-md:text-sm"
- draftSidecars={option.draftSidecars}
- hideCapabilities
- hideModalities
- modalities={option.modalities}
- modelId={option.model}
- tags={option.tags}
- title={option.model}
- />
+ <span class="flex min-w-0 items-center gap-1.25">
+ <ModelId
+ aliases={option.aliases}
+ class="min-w-0 flex-1 max-md:text-sm"
+ draftSidecars={option.draftSidecars}
+ hideCapabilities
+ hideModalities
+ modalities={option.modalities}
+ modelId={option.model}
+ tags={option.tags}
+ title={option.model}
+ />
- <!-- a phone has no width for the modality icons, the id needs it more -->
- <ModelCapabilities hideModalities={deviceStore.isMobile} {option} />
+ <!-- a phone has no width for the modality icons, the id needs it more -->
+ <ModelCapabilities hideModalities={deviceStore.isMobile} {option} />
- {#if favorite}
- <!-- the heart is decorative, the button's text carries the state -->
- <Heart aria-hidden="true" class="h-3.5 w-3.5 shrink-0 text-rose-500" />
+ {#if favorite}
+ <!-- the heart is decorative, the button's text carries the state -->
+ <Heart aria-hidden="true" class="h-3.5 w-3.5 shrink-0 text-rose-500" />
- <span class="sr-only">favorited</span>
- {/if}
- </span>
- </button>
+ <span class="sr-only">favorited</span>
+ {/if}
+ </span>
+ </button>
+
+ {#if sidecarBadges?.length}
+ <!-- w-fit keeps the trailing state beside the badges instead of floating
+ to the end of the wide model column -->
+ <div class="flex w-fit flex-col pb-1">
+ {#each sidecarBadges as badge (badge.repo + ':' + badge.kind + ':' + (badge.quant ?? ''))}
+ <ModelDraftSubline
+ connectorClass="ml-8"
+ kind={badge.kind}
+ quant={badge.quant}
+ state={ModelRowDownloadState.DOWNLOADED}
+ />
+ {/each}
+ </div>
+ {/if}
+ </div>
<ModelContext
class="justify-self-end max-md:hidden"
diff --git a/tools/ui/src/lib/components/app/models/ModelsManager/ModelsManagerModelsTable.svelte b/tools/ui/src/lib/components/app/models/ModelsManager/ModelsManagerModelsTable.svelte
index ae4d7f184..7883cfa7d 100644
--- a/tools/ui/src/lib/components/app/models/ModelsManager/ModelsManagerModelsTable.svelte
+++ b/tools/ui/src/lib/components/app/models/ModelsManager/ModelsManagerModelsTable.svelte
@@ -88,6 +88,7 @@
[GroupedListUnit.GROUPS]: 'families'
};
let pendingDelete = $state('');
+ let pendingAction = $state(ModelDownloadConfirmAction.DELETE);
let deleteOpen = $state(false);
/** Repos whose quants are folded away; the rest show them. */
const collapsedQuants = new SvelteSet<string>();
@@ -129,6 +130,10 @@
// cancel is confirmed once for the whole list, so one dialog serves every row
function requestDelete(option: ModelOption): void {
pendingDelete = option.model;
+ // an in-flight row cancels (and drops the partials), a paused one deletes
+ pendingAction = modelsStore.status.isDownloadInProgress(option.model)
+ ? ModelDownloadConfirmAction.CANCEL
+ : ModelDownloadConfirmAction.DELETE;
deleteOpen = true;
}
@@ -248,6 +253,7 @@
{onSelect}
option={entry.base}
selected={selectedId === entry.base.id}
+ sidecarBadges={entry.sidecarBadges}
/>
{/if}
{/snippet}
@@ -328,7 +334,7 @@
{/snippet}
<DialogConfirmDownload
- action={ModelDownloadConfirmAction.DELETE}
+ action={pendingAction}
onClose={() => (deleteOpen = false)}
open={deleteOpen}
repoWithTag={pendingDelete}
diff --git a/tools/ui/src/lib/components/app/models/ModelsManager/row-actions.ts b/tools/ui/src/lib/components/app/models/ModelsManager/row-actions.ts
index 9931dd3f4..3407634ec 100644
--- a/tools/ui/src/lib/components/app/models/ModelsManager/row-actions.ts
+++ b/tools/ui/src/lib/components/app/models/ModelsManager/row-actions.ts
@@ -1,10 +1,10 @@
-import { Eye, EyeOff, Heart, HeartOff, Trash2 } from '@lucide/svelte';
+import { Compass, Eye, EyeOff, Heart, HeartOff, Trash2 } from '@lucide/svelte';
import { MODEL_DOWNLOAD_ICONS } from '$lib/constants';
import { ModelRowDownloadState } from '$lib/enums';
-import { modelsStore } from '$lib/stores';
+import { modelsStore, uiStore } from '$lib/stores';
import type { ModelOption } from '$lib/types/models';
+import { repoOf } from '$lib/utils';
-/** Row actions follow the app's dropdown pattern: icon, label, separators, variants. */
export function modelRowActions(
option: ModelOption,
favorite: boolean,
@@ -13,9 +13,17 @@ export function modelRowActions(
/** Download state, when the row stands for a tracked download. */
download?: ModelRowDownloadState | null
) {
+ const viewInDiscover = {
+ icon: Compass,
+ label: 'See in Discover',
+ // the details pane is keyed by repo, not by `<repo>:<quant>`
+ onclick: () => uiStore.openModelsDiscover(repoOf(option.model)),
+ separator: true
+ };
+
// a tracked download is paused, resumed or dropped: it is not loaded, hidden or
- // drafted. Deleting it stops the download and removes what is on disk, which is
- // what the same wording offers on a cached model
+ // drafted. An in-flight row cancels, which stops the download and removes the
+ // partial files; a paused one deletes, which removes the partials from disk
if (download) {
const isPaused = download === ModelRowDownloadState.PAUSED;
@@ -23,18 +31,16 @@ export function modelRowActions(
{
icon: isPaused ? MODEL_DOWNLOAD_ICONS.resume : MODEL_DOWNLOAD_ICONS.pause,
label: isPaused ? 'Resume downloading' : 'Pause downloading',
- onclick: () =>
- void (isPaused
- ? modelsStore.status.downloadModel(option.model)
- : modelsStore.status.pauseDownload(option.model))
+ onclick: () => void modelsStore.status.toggleDownload(option.model)
},
{
icon: Trash2,
- label: 'Delete from disk',
+ label: isPaused ? 'Delete from disk' : 'Cancel download',
onclick: () => onDelete(option),
separator: true,
variant: 'destructive' as const
- }
+ },
+ viewInDiscover
];
}
@@ -56,6 +62,7 @@ export function modelRowActions(
label: isHidden ? 'Unhide model' : 'Hide model',
onclick: () => modelsStore.toggleHidden(option.id),
separator: true
- }
+ },
+ viewInDiscover
];
}
diff --git a/tools/ui/src/lib/components/app/models/ModelsManager/utils.ts b/tools/ui/src/lib/components/app/models/ModelsManager/utils.ts
index 208343664..435700a10 100644
--- a/tools/ui/src/lib/components/app/models/ModelsManager/utils.ts
+++ b/tools/ui/src/lib/components/app/models/ModelsManager/utils.ts
@@ -2,7 +2,12 @@ import { LOCAL_BACKEND_ID, type ModalityKey } from '$lib/constants';
import { ModelCapability, ModelGroupKind, ModelsTableGroupKind } from '$lib/enums';
import { HuggingFaceService, ModelsService } from '$lib/services';
import { modelsStore } from '$lib/stores';
-import type { ModelDownloadEntry, ModelDownloadProgress, ModelOption } from '$lib/types/models';
+import type {
+ ModelDownloadEntry,
+ ModelDownloadProgress,
+ ModelOption,
+ ModelSidecarBadge
+} from '$lib/types/models';
import { detectThinkingSupport, detectToolUseSupport, repoOf } from '$lib/utils';
/** One repo of the table, with the rows it ships as. */
@@ -11,6 +16,8 @@ export interface ModelQuantGroup {
key: string;
kind: ModelGroupKind;
quants: ModelOption[];
+ /** Draft sidecars already on disk, shown as checked sub-lines. */
+ sidecarBadges?: ModelSidecarBadge[];
}
/** True when any of the manager's filters narrows the table. */
@@ -31,28 +38,36 @@ export function downloadProgressFor(repoWithTag: string): ModelDownloadProgress
}
/**
- * One table row per tracked download, so the table lists them the way it lists
- * any other model. A paused download is already in the router's listing, so its
- * own option is reused; a fresh one stands for its tag alone.
+ * One table row per tracked download, listed like any other model. A paused download
+ * is already in the router's listing, so its own option is reused; a fresh one is
+ * synthesized for its tag alone.
*/
export function downloadGroups(
entries: ModelDownloadEntry[],
- models: ModelOption[]
+ models: ModelOption[],
+ sidecarsFor: (repo: string) => ModelSidecarBadge[]
): ModelQuantGroup[] {
return entries.map((entry) => {
- const option = models.find((candidate) => candidate.model === entry.repoWithTag) ?? {
+ const listed = models.find((candidate) => candidate.model === entry.repoWithTag);
+ // the synthesized option carries backendId, which ModelOption drops on
+ // assignment, so the callers spread it instead of typing this as one
+ const option = listed ?? {
backendId: LOCAL_BACKEND_ID,
capabilities: [],
id: entry.repoWithTag,
model: entry.repoWithTag,
name: entry.repoWithTag
};
+ // a draft still coming down badges the id, an on-disk one keeps its sub-line
+ const sidecars = sidecarsFor(repoOf(entry.repoWithTag));
+ const pending = sidecars.filter((badge) => badge.pending);
return {
- base: option,
+ base: { ...option, draftSidecars: pending },
key: `download::${entry.repoWithTag}`,
kind: ModelGroupKind.QUANTS,
- quants: [option]
+ quants: [{ ...option, draftSidecars: pending }],
+ sidecarBadges: sidecars.filter((badge) => !badge.pending)
};
});
}
diff --git a/tools/ui/src/lib/components/app/models/ModelsSelector/ModelsSelectorDownloadItem.svelte b/tools/ui/src/lib/components/app/models/ModelsSelector/ModelsSelectorDownloadItem.svelte
index 6ef1433b9..7cc286f89 100644
--- a/tools/ui/src/lib/components/app/models/ModelsSelector/ModelsSelectorDownloadItem.svelte
+++ b/tools/ui/src/lib/components/app/models/ModelsSelector/ModelsSelectorDownloadItem.svelte
@@ -1,22 +1,38 @@
<script lang="ts">
import ModelDownloadProgressBar from '../ModelDownloadProgressBar.svelte';
+ import ModelDraftSubline from '../ModelDraftSubline.svelte';
import ModelOrgAvatar from '../ModelOrgAvatar.svelte';
- import { Loader2, Pause, Play, X } from '@lucide/svelte';
+ import { Info, Loader2, Pause, Play, X } from '@lucide/svelte';
import { ActionIcon, ModelId } from '$lib/components/app';
+ import { ModelRowDownloadState } from '$lib/enums';
import { HuggingFaceService, ModelsService } from '$lib/services';
- import { modelsStore } from '$lib/stores';
- import type { ModelDownloadEntry } from '$lib/types';
+ import { deviceStore, modelsStore, uiStore } from '$lib/stores';
+ import type { ModelDownloadEntry, ModelOption, ModelSidecarBadge } from '$lib/types';
interface Props {
/** One entry from the status feed: an in-flight or paused download. */
entry: ModelDownloadEntry;
+ /** Draft sidecars already on disk, rendered as checked sub-lines. */
+ sidecars?: ModelSidecarBadge[];
/** Ask the list to confirm cancelling; the list owns the single dialog. */
onRequestCancel?: (repoWithTag: string) => void;
/** Show the organization name in the repo id, as the other selector rows do. */
showOrgName?: boolean;
}
- let { entry, onRequestCancel, showOrgName = true }: Props = $props();
+ let { entry, onRequestCancel, showOrgName = true, sidecars = [] }: Props = $props();
+
+ /** A draft still coming down badges the id, an on-disk one keeps its sub-line. */
+ let pendingDrafts = $derived(sidecars.filter((badge) => badge.pending));
+ let onDiskDrafts = $derived(sidecars.filter((badge) => !badge.pending));
+
+ /** The download stands for its model, so the row's info action can open on it. */
+ let option = $derived<ModelOption>({
+ capabilities: [],
+ id: entry.repoWithTag,
+ model: entry.repoWithTag,
+ name: modelsStore.toDisplayName(entry.repoWithTag)
+ });
let percent = $derived(
entry.progress && entry.progress.totalBytes > 0
@@ -53,48 +69,60 @@
});
</script>
-<!-- One in-flight download; same actions as the discover quant chips: the row
- itself pauses / resumes, the trailing X cancels (stops and discards the
- partial files). Both affordances fade in on hover, the slots are reserved
- so the list never reflows. -->
-<div
- class="group relative flex items-center gap-2 rounded-sm p-2 text-left text-sm transition-colors hover:bg-accent/50"
->
- <button
- aria-label={actionText}
- class="flex min-w-0 flex-1 cursor-pointer items-center gap-2 text-left"
- onclick={() => {
- if (entry.isPaused) {
- void modelsStore.status
- .downloadModel(entry.repoWithTag)
- .catch((error) => console.warn('Failed to resume the download:', error));
- } else {
- void modelsStore.status.pauseDownload(entry.repoWithTag);
- }
- }}
- type="button"
+<!-- One in-flight download: the row opens the model, the status slot beside it
+ pauses / resumes and the trailing X cancels (stops and discards the partial
+ files). The status slot is always there, the affordances fade in on hover,
+ so the list never reflows. The repo's on-disk sidecars hang off the item as
+ compact sub-lines instead of rows of their own. -->
+<div class="group relative rounded-sm transition-colors hover:bg-accent/50">
+ <div
+ class="flex items-center gap-2 p-2 text-left text-sm max-md:gap-2.5 max-md:px-2.5 max-md:py-2.5"
>
- {#if orgName}
- <ModelOrgAvatar
- org={fetchedBaseModelOrg ?? orgName}
- quantOrg={orgName}
- quantPositionClass="-bottom-1 -right-1"
- quantSize="h-3 w-3"
- size="h-6 w-6"
- />
- {/if}
+ <button
+ aria-label="Manage model"
+ class="flex min-w-0 flex-1 cursor-pointer items-center gap-2 text-left"
+ onclick={() =>
+ // a phone has no manager: its information dialog takes the row
+ deviceStore.isMobile
+ ? uiStore.openModelInformation(option)
+ : uiStore.openModelsManager(entry.repoWithTag)}
+ type="button"
+ >
+ {#if orgName}
+ <ModelOrgAvatar
+ org={fetchedBaseModelOrg ?? orgName}
+ quantOrg={orgName}
+ quantPositionClass="-bottom-1 -right-1"
+ quantSize="h-3 w-3"
+ size="h-6 w-6"
+ />
+ {/if}
- <ModelId class="flex-1" hideOrgName={!showOrgName} modelId={entry.repoWithTag} showRawTooltip />
+ <ModelId
+ class="flex-1"
+ draftSidecars={pendingDrafts}
+ hideOrgName={!showOrgName}
+ modelId={entry.repoWithTag}
+ showRawTooltip
+ />
- {#if percent !== null}
- <span class="shrink-0 font-mono text-xs tabular-nums text-muted-foreground">{percent}%</span>
- {:else if entry.isPaused}
- <span class="shrink-0 text-xs text-muted-foreground">Paused</span>
- {/if}
+ {#if percent !== null}
+ <span class="ml-auto shrink-0 font-mono text-xs tabular-nums text-muted-foreground">
+ {percent}%
+ </span>
+ {:else if entry.isPaused}
+ <span class="ml-auto shrink-0 text-xs text-muted-foreground">Paused</span>
+ {/if}
+ </button>
<!-- status action: spinner -> pause on hover while in flight, play on hover
- when paused; opacity only for the spinner, the spin owns the transform -->
- <span class="relative inline-flex h-5 w-5 shrink-0 items-center justify-center">
+ when paused; opacity only for the spinner, the spin owns the transform -->
+ <button
+ aria-label={actionText}
+ class="relative inline-flex h-5 w-5 shrink-0 cursor-pointer items-center justify-center"
+ onclick={() => void modelsStore.status.toggleDownload(entry.repoWithTag)}
+ type="button"
+ >
{#if entry.isPaused}
<Play
class="absolute h-4 w-4 scale-75 opacity-0 transition-[opacity,transform] duration-150 ease-[cubic-bezier(0.23,1,0.32,1)] group-hover:scale-100 group-hover:opacity-100 [@media(pointer:coarse)]:scale-100 [@media(pointer:coarse)]:opacity-100"
@@ -108,17 +136,45 @@
class="absolute h-4 w-4 opacity-0 transition-opacity duration-150 ease-[cubic-bezier(0.23,1,0.32,1)] group-hover:opacity-100 [@media(pointer:coarse)]:opacity-100"
/>
{/if}
- </span>
- </button>
-
- <!-- the row actions keep the sizing and reveal of the other selector rows -->
- <ActionIcon
- ariaLabel="Cancel downloading"
- class="pointer-events-none h-5 w-5 shrink-0 opacity-0 transition-opacity group-hover:pointer-events-auto group-hover:opacity-100 hover:text-destructive [@media(pointer:coarse)]:pointer-events-auto [@media(pointer:coarse)]:opacity-100"
- icon={X}
- iconSize="h-4 w-4"
- onclick={() => onRequestCancel?.(entry.repoWithTag)}
- />
+ </button>
+
+ <!-- the row actions keep the sizing and reveal of the other selector rows -->
+ <ActionIcon
+ class="pointer-events-none h-5 w-5 shrink-0 opacity-0 transition-opacity group-hover:pointer-events-auto group-hover:opacity-100 hover:text-foreground [@media(pointer:coarse)]:pointer-events-auto [@media(pointer:coarse)]:opacity-100"
+ icon={Info}
+ iconSize="h-4 w-4"
+ onclick={() =>
+ // a phone has no manager: its information dialog takes the icon
+ deviceStore.isMobile
+ ? uiStore.openModelInformation(option)
+ : uiStore.openModelsManager(entry.repoWithTag)}
+ tooltip="Manage model"
+ tooltipAsTitle
+ />
+
+ <ActionIcon
+ ariaLabel="Cancel downloading"
+ class="pointer-events-none h-5 w-5 shrink-0 opacity-0 transition-opacity group-hover:pointer-events-auto group-hover:opacity-100 hover:text-destructive [@media(pointer:coarse)]:pointer-events-auto [@media(pointer:coarse)]:opacity-100"
+ icon={X}
+ iconSize="h-4 w-4"
+ onclick={() => onRequestCancel?.(entry.repoWithTag)}
+ />
+ </div>
+
+ <!-- only with sub-lines: an empty strip would make the row taller than the
+ plain ones, and the progress bar overlays the row's own bottom edge -->
+ {#if onDiskDrafts.length}
+ <div class="flex flex-col px-2 pb-1.5">
+ {#each onDiskDrafts as badge (badge.repo + ':' + badge.kind + ':' + (badge.quant ?? ''))}
+ <ModelDraftSubline
+ connectorClass="ml-4"
+ kind={badge.kind}
+ quant={badge.quant}
+ state={ModelRowDownloadState.DOWNLOADED}
+ />
+ {/each}
+ </div>
+ {/if}
{#if entry.progress && entry.progress.totalBytes > 0}
<ModelDownloadProgressBar
diff --git a/tools/ui/src/lib/components/app/models/ModelsSelector/ModelsSelectorList.svelte b/tools/ui/src/lib/components/app/models/ModelsSelector/ModelsSelectorList.svelte
index 6857e3018..29adb3c04 100644
--- a/tools/ui/src/lib/components/app/models/ModelsSelector/ModelsSelectorList.svelte
+++ b/tools/ui/src/lib/components/app/models/ModelsSelector/ModelsSelectorList.svelte
@@ -22,6 +22,7 @@
groupModelFamilies,
type ModelFamilyGroup,
type ModelItem,
+ repoOf,
windowLocalGroups
} from '$lib/utils';
@@ -202,7 +203,12 @@
<p class={headerClass} style={headerStyle}>Downloading</p>
{#each downloadEntries as entry (entry.repoWithTag)}
- <ModelsSelectorDownloadItem {entry} onRequestCancel={requestCancel} {showOrgName} />
+ <ModelsSelectorDownloadItem
+ {entry}
+ onRequestCancel={requestCancel}
+ {showOrgName}
+ sidecars={modelsStore.status.getDraftSidecars(repoOf(entry.repoWithTag))}
+ />
{/each}
{/if}
@@ -230,6 +236,7 @@
<DialogConfirmDownload
action={ModelDownloadConfirmAction.CANCEL}
onClose={() => (cancelOpen = false)}
+ onConfirm={() => void modelsStore.status.cancelDownload(pendingCancel)}
open={cancelOpen}
repoWithTag={pendingCancel}
/>
diff --git a/tools/ui/src/lib/components/app/models/discover/ModelsDiscover.svelte b/tools/ui/src/lib/components/app/models/discover/ModelsDiscover.svelte
new file mode 100644
index 000000000..292938d59
--- /dev/null
+++ b/tools/ui/src/lib/components/app/models/discover/ModelsDiscover.svelte
@@ -0,0 +1,210 @@
+<script lang="ts">
+ import ModelsDiscoverListItem from './ModelsDiscoverList/ModelsDiscoverListItem.svelte';
+ import { Download } from '@lucide/svelte';
+ import { ModelsSection } from '$lib/components/app';
+ import { DialogConfirmDownload } from '$lib/components/app/dialogs';
+ import {
+ ModelsDiscoverDetailsContainer,
+ ModelsDiscoverList,
+ ModelsDiscoverListSearch
+ } from '$lib/components/app/models/discover';
+ import { Button } from '$lib/components/ui/button';
+ import { ModelDownloadConfirmAction } from '$lib/enums';
+ import { HuggingFaceService } from '$lib/services';
+ import { modelsDiscoverStore, modelsStore, uiStore } from '$lib/stores';
+ import type { HfModelInfo } from '$lib/types';
+ import { repoOf } from '$lib/utils';
+ import { untrack } from 'svelte';
+
+ let selectedId = $state<string | null>(null);
+ let searchQuery = $state('');
+ // running and paused downloads both lead the list; a search owns the pane, so the
+ // section steps aside while a query is active
+ let downloadEntries = $derived(modelsStore.status.getDownloadEntries());
+
+ let isSearching = $derived(searchQuery.trim().length > 0);
+
+ /** Hub record for a repo the list does not carry, so its row still renders. */
+ function hubRecord(id: string): HfModelInfo {
+ return {
+ _id: id,
+ downloads: 0,
+ id,
+ library_name: null,
+ likes: 0,
+ modelId: id,
+ pipeline_tag: null,
+ private: false,
+ tags: []
+ };
+ }
+
+ // Hub records of repos the list does not carry, e.g. a download started from a
+ // search hit: fetched once so the row shows the same badges and icons as a result.
+ let fetchedRecords = $state<Record<string, HfModelInfo>>({});
+
+ /** Record of a repo: what the list fetched, else its own details, else a bare one. */
+ function recordOf(repo: string): HfModelInfo {
+ return (
+ modelsDiscoverStore.models.find((model) => model.id === repo) ??
+ fetchedRecords[repo] ??
+ hubRecord(repo)
+ );
+ }
+
+ $effect(() => {
+ for (const entry of downloadEntries) {
+ const repo = repoOf(entry.repoWithTag);
+
+ if (fetchedRecords[repo] || modelsDiscoverStore.models.some((m) => m.id === repo)) continue;
+
+ void HuggingFaceService.getDetails(repo).then((details) => {
+ if (details) fetchedRecords = { ...fetchedRecords, [repo]: details };
+ });
+ }
+ });
+
+ // cancel is confirmed once, so a single dialog serves every download row; the
+ // target is kept while the dialog closes so its copy stays rendered
+ let pendingCancel = $state('');
+ let cancelOpen = $state(false);
+
+ function requestCancel(repoWithTag: string) {
+ pendingCancel = repoWithTag;
+ cancelOpen = true;
+ }
+
+ let searchTimeout: ReturnType<typeof setTimeout> | null = null;
+
+ // load the list on mount; untrack because fetch() toggles its loading flag
+ // synchronously, and tracking it would re-run the effect on every toggle - an
+ // endless retry while a repo stays failed
+ $effect(() => {
+ untrack(() => {
+ void modelsDiscoverStore.fetch();
+ void modelsDiscoverStore.search('');
+ });
+ });
+
+ $effect(() => {
+ const first = modelsDiscoverStore.firstModel;
+
+ if (!selectedId && first) {
+ selectedId = first.id;
+ }
+ });
+
+ // a caller can ask for one repo to be opened, the manager's rows do
+ $effect(() => {
+ const focus = uiStore.discoverModelFocus;
+
+ if (!focus) return;
+
+ selectedId = focus;
+ uiStore.discoverModelFocus = null;
+ });
+
+ function handleSearchInput(value: string) {
+ searchQuery = value;
+
+ if (searchTimeout) clearTimeout(searchTimeout);
+
+ searchTimeout = setTimeout(() => {
+ void modelsDiscoverStore.search(value);
+ }, 300);
+ }
+</script>
+
+{#snippet catalog()}
+ <ModelsDiscoverList
+ activeId={selectedId}
+ loading={modelsDiscoverStore.loading || modelsDiscoverStore.searching}
+ models={modelsDiscoverStore.models}
+ onSelect={(id) => (selectedId = id)}
+ showBaseModelAvatar
+ />
+{/snippet}
+
+<aside
+ class="w-md shrink-0 self-start border-r border-border/40 bg-background overflow-y-auto md:pr-4 h-full space-y-4"
+>
+ <ModelsDiscoverListSearch bind:value={searchQuery} onSearch={handleSearchInput} />
+
+ {#if downloadEntries.length > 0 && !isSearching}
+ <ModelsSection
+ count={downloadEntries.length}
+ label="Downloading"
+ persistKey="discover-downloads"
+ sticky
+ stickyClass="sticky z-10 bg-background"
+ >
+ {#snippet icon()}
+ <Download class="h-3.5 w-3.5 shrink-0" />
+ {/snippet}
+
+ {#each downloadEntries as entry (entry.repoWithTag)}
+ {@const repo = repoOf(entry.repoWithTag)}
+
+ <!-- the row is the list's own, so a download opens the same details pane a
+ result does; only the state line and the trailing control differ -->
+ <ModelsDiscoverListItem
+ active={repo === selectedId}
+ download={{
+ isPaused: entry.isPaused,
+ onRequestCancel: requestCancel,
+ progress: entry.progress,
+ repoWithTag: entry.repoWithTag
+ }}
+ model={recordOf(repo)}
+ onSelect={(id) => (selectedId = id)}
+ showBaseModelAvatar
+ sidecarBadges={modelsStore.status.getDraftSidecars(repo)}
+ />
+ {/each}
+ </ModelsSection>
+ {/if}
+
+ <!-- One list instance, so the rows keep their state across search round trips;
+ skeleton rows replace them while the initial catalog or a query loads. -->
+ <div>
+ {#if modelsDiscoverStore.error}
+ <div class="flex flex-col items-start gap-2 p-4">
+ <p class="text-sm text-destructive">{modelsDiscoverStore.error}</p>
+
+ <Button onclick={() => void modelsDiscoverStore.fetch()} size="sm" variant="outline">
+ Retry
+ </Button>
+ </div>
+ {:else if !modelsDiscoverStore.loading && !modelsDiscoverStore.searching && modelsDiscoverStore.models.length === 0}
+ <p class="p-4 text-sm text-muted-foreground">No models found</p>
+ {:else}
+ {#if isSearching}
+ {@render catalog()}
+ {:else}
+ <ModelsSection
+ count={modelsDiscoverStore.models.length}
+ label="Suggested models"
+ persistKey="discover-suggested"
+ sticky
+ stickyClass="sticky z-10 bg-background"
+ >
+ {@render catalog()}
+ </ModelsSection>
+ {/if}
+ {/if}
+ </div>
+</aside>
+
+<main class="overflow-y-auto">
+ {#if selectedId}
+ <ModelsDiscoverDetailsContainer modelId={selectedId} />
+ {/if}
+</main>
+
+<DialogConfirmDownload
+ action={ModelDownloadConfirmAction.CANCEL}
+ onClose={() => (cancelOpen = false)}
+ onConfirm={() => void modelsStore.status.cancelDownload(pendingCancel)}
+ open={cancelOpen}
+ repoWithTag={pendingCancel}
+/>
diff --git a/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverChatTemplateDialog.svelte b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverChatTemplateDialog.svelte
new file mode 100644
index 000000000..11240a4a9
--- /dev/null
+++ b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverChatTemplateDialog.svelte
@@ -0,0 +1,75 @@
+<script lang="ts">
+ import { Check, Copy, X } from '@lucide/svelte';
+ import { SyntaxHighlightedCode } from '$lib/components/app';
+ import { Button } from '$lib/components/ui/button';
+ import * as Dialog from '$lib/components/ui/dialog';
+ import { COPY_FEEDBACK_RESET_MS } from '$lib/constants';
+ import { copyToClipboard } from '$lib/utils';
+
+ interface Props {
+ open?: boolean;
+ chatTemplate: string;
+ onOpenChange?: (open: boolean) => void;
+ }
+
+ let { chatTemplate, onOpenChange, open = $bindable(false) }: Props = $props();
+
+ function handleOpenChange(value: boolean) {
+ open = value;
+ onOpenChange?.(value);
+ }
+
+ let copied = $state(false);
+
+ async function copy() {
+ await copyToClipboard(chatTemplate);
+ copied = true;
+ setTimeout(() => (copied = false), COPY_FEEDBACK_RESET_MS);
+ }
+</script>
+
+<Dialog.Root onOpenChange={handleOpenChange} {open}>
+ <Dialog.Content
+ class="flex max-h-[calc(100vh-4rem)] flex-col gap-0 p-0 md:w-[calc(100vw-4rem)]! md:max-w-4xl!"
+ >
+ <!-- The header's corner-pinned close X never lines up with a padded flex row,
+ so it is replaced by one inside the row, aligned with the title -->
+ <Dialog.Header
+ class="flex-row items-center gap-2 border-b border-border/40 p-4"
+ showCloseButton={false}
+ >
+ <Dialog.Title class="text-sm font-semibold">Chat template</Dialog.Title>
+
+ <Button
+ aria-label="Copy chat template"
+ class="h-auto gap-1.5 px-2 py-1 text-xs"
+ onclick={() => void copy()}
+ size="sm"
+ variant="tertiary"
+ >
+ {#if copied}
+ <Check class="size-3.5 text-green-500" />
+ {:else}
+ <Copy class="size-3.5" />
+ {/if}
+
+ Copy
+ </Button>
+
+ <Dialog.Close
+ aria-label="Close"
+ class="ml-auto inline-flex cursor-pointer items-center justify-center rounded-md p-1.5 text-muted-foreground/70 transition-colors hover:bg-muted-foreground/10 hover:text-foreground"
+ type="button"
+ >
+ <X class="h-4 w-4" />
+ </Dialog.Close>
+ </Dialog.Header>
+
+ <SyntaxHighlightedCode
+ class="h-full rounded-none border-0 shadow-none"
+ code={chatTemplate}
+ language="jinja"
+ maxHeight="100%"
+ />
+ </Dialog.Content>
+</Dialog.Root>
diff --git a/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetails.svelte b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetails.svelte
new file mode 100644
index 000000000..8f7e50da6
--- /dev/null
+++ b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetails.svelte
@@ -0,0 +1,96 @@
+<script lang="ts">
+ import { ModelsDiscoverDetailsDownloadOptions } from './ModelsDiscoverDetailsDownloadOptions';
+ import ModelsDiscoverDetailsHeader from './ModelsDiscoverDetailsHeader.svelte';
+ import ModelsDiscoverDetailsReadme from './ModelsDiscoverDetailsReadme.svelte';
+ import ModelsDiscoverDetailsSkeleton from './ModelsDiscoverDetailsSkeleton.svelte';
+ import { OTHER_BIT_DEPTH } from '$lib/constants';
+ import { ModelAuxSidecar } from '$lib/enums';
+ import { HuggingFaceService } from '$lib/services';
+ import type { HfModelDetailInfo, HfModelSibling, ModelBitDepthRow } from '$lib/types';
+ import { detectThinkingSupport, detectToolUseSupport } from '$lib/utils';
+ import { SvelteMap } from 'svelte/reactivity';
+
+ interface Props {
+ /** Full HuggingFace model id, e.g. `ggml-org/gemma-3-4b-it-GGUF`. */
+ modelId: string;
+ /** Model details from `/api/models/{id}?full=true`; null while loading. */
+ details: HfModelDetailInfo | null;
+ /** GGUF files of the repo, shards collapsed, sorted by size desc. */
+ files: HfModelSibling[];
+ /** README.md content, frontmatter stripped; null when unavailable. */
+ readme: string | null;
+ /** True while the model data is being fetched. */
+ loading?: boolean;
+ /** Error message when loading failed. */
+ error?: string | null;
+ }
+
+ let { details, error = null, files, loading = false, modelId, readme }: Props = $props();
+
+ let gguf = $derived(details?.gguf);
+ let baseModels = $derived(HuggingFaceService.getBaseModels(details));
+ let licenseTag = $derived.by(() => {
+ const tags = details?.tags ?? [];
+
+ return tags.find((t) => t.startsWith('license:'))?.replace('license:', '') ?? null;
+ });
+
+ // Capabilities derived from HF metadata. Vision comes from an mmproj sidecar
+ // or a multimodal pipeline tag; tool use / reasoning from the chat template.
+ let hasMmproj = $derived(
+ files.some(
+ (f) => HuggingFaceService.extractQuantMeta(f.path)?.sidecar === ModelAuxSidecar.MMPROJ
+ )
+ );
+ let hasVision = $derived(hasMmproj || details?.pipeline_tag === 'image-text-to-text');
+ let hasTools = $derived(detectToolUseSupport(gguf?.chat_template ?? ''));
+ let hasReasoning = $derived(detectThinkingSupport(gguf?.chat_template ?? ''));
+
+ let bitDepthRows = $derived.by<ModelBitDepthRow[]>(() => {
+ const rows = new SvelteMap<number, HfModelSibling[]>();
+
+ for (const file of files) {
+ const meta = HuggingFaceService.extractQuantMeta(file.path);
+
+ // mmproj sidecars are already conveyed by the Vision capability badge;
+ // imatrix ships as a normal chip with its own badge.
+ if (meta?.sidecar === ModelAuxSidecar.MMPROJ) continue;
+
+ const depth = meta?.quant ? HuggingFaceService.getBitDepth(meta.quant) : null;
+ const bucket = depth ?? OTHER_BIT_DEPTH;
+ const list = rows.get(bucket) ?? [];
+
+ list.push(file);
+ rows.set(bucket, list);
+ }
+
+ return Array.from(rows.entries())
+ .map(([bitDepth, rowFiles]) => ({ bitDepth, files: rowFiles }))
+ .sort((a, b) => a.bitDepth - b.bitDepth);
+ });
+</script>
+
+{#if loading}
+ <ModelsDiscoverDetailsSkeleton />
+{:else if error}
+ <div class="flex h-full items-center justify-center py-20">
+ <p class="text-sm text-destructive">{error}</p>
+ </div>
+{:else if details}
+ <div class="@container space-y-6 p-6 pr-0">
+ <ModelsDiscoverDetailsHeader
+ {baseModels}
+ {details}
+ {gguf}
+ {hasReasoning}
+ {hasTools}
+ {hasVision}
+ {licenseTag}
+ {modelId}
+ />
+
+ <ModelsDiscoverDetailsDownloadOptions {bitDepthRows} {modelId} />
+
+ <ModelsDiscoverDetailsReadme {readme} />
+ </div>
+{/if}
diff --git a/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsContainer.svelte b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsContainer.svelte
new file mode 100644
index 000000000..7d81ea8ab
--- /dev/null
+++ b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsContainer.svelte
@@ -0,0 +1,66 @@
+<script lang="ts">
+ import ModelsDiscoverDetails from './ModelsDiscoverDetails.svelte';
+ import { HuggingFaceService } from '$lib/services';
+ import type { HfModelDetailInfo, HfModelSibling } from '$lib/types/huggingface';
+
+ interface Props {
+ /** Full HuggingFace model id, e.g. `ggml-org/gemma-3-4b-it-GGUF`. */
+ modelId: string;
+ }
+
+ let { modelId }: Props = $props();
+
+ let details = $state<HfModelDetailInfo | null>(null);
+ let files = $state<HfModelSibling[]>([]);
+ let readme = $state<string | null>(null);
+ let loading = $state(false);
+ let error = $state<string | null>(null);
+
+ $effect(() => {
+ const id = modelId;
+
+ if (!id) return;
+
+ let cancelled = false;
+
+ loading = true;
+ error = null;
+
+ void (async () => {
+ try {
+ const [info, tree, readmeText] = await Promise.all([
+ HuggingFaceService.getDetails(id),
+ HuggingFaceService.getTree(id),
+ HuggingFaceService.getReadme(id)
+ ]);
+
+ if (cancelled) return;
+
+ if (!info) {
+ error = 'Model not found';
+
+ return;
+ }
+
+ details = info;
+ files = HuggingFaceService.filterByExtension(
+ HuggingFaceService.collapseGgufShards(tree),
+ '.gguf'
+ );
+ readme = readmeText;
+ } catch (err) {
+ if (cancelled) return;
+
+ error = err instanceof Error ? err.message : 'Failed to load model';
+ } finally {
+ if (!cancelled) loading = false;
+ }
+ })();
+
+ return () => {
+ cancelled = true;
+ };
+ });
+</script>
+
+<ModelsDiscoverDetails {details} {error} {files} {loading} {modelId} {readme} />
diff --git a/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsDownloadOptions/ModelsDiscoverDetailsDownloadOptions.svelte b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsDownloadOptions/ModelsDiscoverDetailsDownloadOptions.svelte
new file mode 100644
index 000000000..a1bd05a37
--- /dev/null
+++ b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsDownloadOptions/ModelsDiscoverDetailsDownloadOptions.svelte
@@ -0,0 +1,258 @@
+<script lang="ts">
+ import { classify, draftBadgeOf, sortDraftKinds } from './download-options.utils';
+ import ModelsDiscoverDetailsDownloadOptionsRow from './ModelsDiscoverDetailsDownloadOptionsRow.svelte';
+ import { DialogConfirmDownload } from '$lib/components/app/dialogs';
+ import { Checkbox } from '$lib/components/ui/checkbox';
+ import * as Select from '$lib/components/ui/select';
+ import {
+ ModelDownloadConfirmAction,
+ ModelDraftSidecar,
+ ModelSelectableFileKind
+ } from '$lib/enums';
+ import { HuggingFaceService, ModelsService } from '$lib/services';
+ import { modelsStore } from '$lib/stores';
+ import type {
+ ModelBitDepthRow,
+ ModelCompanionDraft,
+ ModelDownloadEntryState,
+ ModelSelectableFile
+ } from '$lib/types';
+ import { directoryOf, isServerSidecarFile } from '$lib/utils';
+
+ interface Props {
+ /** Full HuggingFace repo id, e.g. `ggml-org/gemma-3-4b-it-GGUF`. */
+ modelId: string;
+ /** GGUF files grouped by bit depth. */
+ bitDepthRows: ModelBitDepthRow[];
+ }
+
+ let { bitDepthRows, modelId }: Props = $props();
+
+ // one shared dialog confirms the panel's destructive chip actions; the target
+ // is kept after closing so its copy stays rendered through the close transition
+ let pending: { action: ModelDownloadConfirmAction; repoWithTag: string } = $state({
+ action: ModelDownloadConfirmAction.CANCEL,
+ repoWithTag: ''
+ });
+ let confirmOpen = $state(false);
+
+ function requestCancel(repoWithTag: string) {
+ pending = { action: ModelDownloadConfirmAction.CANCEL, repoWithTag };
+ confirmOpen = true;
+ }
+
+ function requestDelete(repoWithTag: string) {
+ pending = { action: ModelDownloadConfirmAction.DELETE, repoWithTag };
+ confirmOpen = true;
+ }
+
+ /**
+ * Download state of a main quant's chip, from the status feed: the quant alone,
+ * since a draft it is missing gets a chip of its own to add it.
+ */
+ function stateFor(repoWithTag: string): ModelDownloadEntryState {
+ const isDownloading = modelsStore.status.isDownloadInProgress(repoWithTag);
+ const isPaused = modelsStore.status.isDownloadPaused(repoWithTag);
+
+ return {
+ isDownloaded: !isDownloading && modelsStore.status.isModelDownloaded(repoWithTag),
+ isDownloading,
+ isFailed: modelsStore.status.hasFailedDownload(repoWithTag),
+ isPaused,
+ // live progress while downloading, else the frozen snapshot of the pause
+ progress:
+ modelsStore.status.getDownloadProgress(repoWithTag) ??
+ modelsStore.status.getPausedDownloadProgress(repoWithTag),
+ repoWithTag
+ };
+ }
+
+ /**
+ * Every GGUF of the panel with its role, in row order.
+ * Single source of truth for the chip rows and the command selects.
+ */
+ let selectableFiles = $derived.by(() =>
+ bitDepthRows.flatMap((row) => row.files.map((file) => ({ ...file, kind: classify(file.path) })))
+ );
+
+ /** Sidecar kinds the repo reports on disk, so a chip tells a pulled draft from a missing one. */
+ let onDiskDrafts = $derived(modelsStore.status.getDraftSidecars(modelId));
+
+ /** Draft kinds the repo ships, best first and once each; one rides with every chip. */
+ let draftKinds = $derived(
+ sortDraftKinds(
+ selectableFiles
+ .flatMap((file) =>
+ file.kind === ModelSelectableFileKind.DRAFT ? [draftBadgeOf(file)] : []
+ )
+ .filter((kind): kind is ModelDraftSidecar => kind !== null)
+ .filter((kind, index, all) => all.indexOf(kind) === index)
+ )
+ );
+
+ // the operator's draft pick; the best available kind rides along until they change it
+ let draftPick = $state<ModelDraftSidecar | null>(null);
+ let draftKind = $derived(draftPick ?? draftKinds[0] ?? null);
+
+ /** The drafts are opt-in: unchecked, a chip downloads the quant alone. */
+ let withDraft = $state(true);
+
+ /** One kind leaves the pick nothing to choose: it reads as the value it holds. */
+ let draftKindReadonly = $derived(draftKinds.length < 2);
+
+ let draftKindLabel = $derived(draftKind?.toUpperCase() ?? '');
+
+ /** Bits of a file's quant, 0 when its name carries none, as the server reads it. */
+ function bitsOf(path: string): number {
+ const quant = HuggingFaceService.extractQuantMeta(path)?.quant;
+
+ return (quant ? HuggingFaceService.getBitDepth(quant) : null) ?? 0;
+ }
+
+ /**
+ * Draft that rides with a quant: the file of the picked kind the server would
+ * pull. Its own directory, then the exact quant, then the closest bit depth -
+ * a 4-bit model pairs with a 16-bit draft head when the repo ships no 4-bit
+ * one, the way -hf settles it.
+ */
+ function companionFor(
+ main: Pick<ModelSelectableFile, 'path'>,
+ quant: string | null
+ ): ModelCompanionDraft | null {
+ if (!draftKind) return null;
+
+ const directory = directoryOf(main.path);
+ const candidates = selectableFiles.filter(
+ (candidate) =>
+ candidate.kind === ModelSelectableFileKind.DRAFT &&
+ draftBadgeOf(candidate) === draftKind &&
+ isServerSidecarFile(candidate.path, draftKind) &&
+ directoryOf(candidate.path) === directory
+ );
+
+ if (candidates.length === 0) return null;
+
+ const quantOf = (candidate: ModelSelectableFile) =>
+ HuggingFaceService.extractQuantMeta(candidate.path)?.quant ?? null;
+ const mainBits = quant ? bitsOf(main.path) : 0;
+ const exact = quant ? candidates.find((candidate) => quantOf(candidate) === quant) : undefined;
+ // the closest bit depth wins when the exact quant is missing, ties on file order
+ const closest = [...candidates].sort(
+ (a, b) => Math.abs(bitsOf(a.path) - mainBits) - Math.abs(bitsOf(b.path) - mainBits)
+ )[0];
+ const picked = exact ?? closest;
+
+ if (!picked) return null;
+
+ const pickedQuant = quantOf(picked);
+
+ return {
+ kind: draftKind,
+ // on disk when the model's own sidecar option reports it, or when a loaded
+ // model's args name that same draft file
+ onDisk:
+ modelsStore.status.isModelDownloaded(
+ ModelsService.buildDownloadTag(modelId, quant, draftKind)
+ ) || onDiskDrafts.some((badge) => badge.kind === draftKind && badge.quant === pickedQuant),
+ path: picked.path,
+ quant: pickedQuant,
+ size: picked.size ?? 0
+ };
+ }
+
+ let rows = $derived.by(() =>
+ bitDepthRows.map((row) => ({
+ bitDepth: row.bitDepth,
+ files: row.files.flatMap((file) => {
+ if (classify(file.path) !== ModelSelectableFileKind.MAIN) return [];
+
+ const quant = HuggingFaceService.extractQuantMeta(file.path)?.quant ?? null;
+ const companion = withDraft ? companionFor(file, quant) : null;
+
+ return [
+ {
+ ...file,
+ companion,
+ kind: ModelSelectableFileKind.MAIN,
+ quant,
+ // the store adds the model's mmproj and the sidecars it already pulls
+ sidecars: companion ? [companion.kind] : [],
+ state: stateFor(ModelsService.buildDownloadTag(modelId, quant, null))
+ }
+ ];
+ })
+ }))
+ );
+</script>
+
+{#if bitDepthRows.length}
+ <section class="rounded-3xl border border-border/30 bg-muted/60 shadow-sm dark:border-border/20">
+ <!-- One chip per main quant, each an independent download action. The draft
+ of that quant rides with it and badges the chip; the server picks its file,
+ the way -hf does. -->
+ <div class="flex w-full flex-col divide-y divide-border/50 px-4 pb-1 dark:divide-border/35">
+ {#each rows as row (row.bitDepth)}
+ <ModelsDiscoverDetailsDownloadOptionsRow
+ bitDepth={row.bitDepth}
+ draftKind={withDraft ? draftKind : null}
+ {draftKinds}
+ files={row.files}
+ {modelId}
+ onDraftKindChange={(kind) => (draftPick = kind)}
+ onRequestCancel={requestCancel}
+ onRequestDelete={requestDelete}
+ />
+ {/each}
+ </div>
+
+ <!-- The drafts are opt-in, and the pick decides which flavour every chip
+ above pulls. The pick sits between the two label halves, so its own click
+ cannot toggle the checkbox. A repo shipping a single kind still shows it,
+ readonly. -->
+ {#if draftKinds.length}
+ <div
+ class="flex flex-wrap items-center justify-start gap-x-1.5 gap-y-1 border-t border-border/50 px-4 py-2.5 text-xs text-muted-foreground dark:border-border/35"
+ >
+ <label class="flex cursor-pointer items-center gap-2" for="discover-draft-sidecars">
+ <Checkbox bind:checked={withDraft} id="discover-draft-sidecars" />
+
+ Download model with a
+ </label>
+
+ <Select.Root
+ disabled={!withDraft || draftKindReadonly}
+ onValueChange={(value) => value && (draftPick = value as ModelDraftSidecar)}
+ type="single"
+ value={draftKind}
+ >
+ <Select.Trigger
+ aria-label="Draft type"
+ class={draftKindReadonly
+ ? 'disabled:cursor-default disabled:opacity-100 [&_svg]:hidden'
+ : ''}
+ size="xs"
+ >
+ {draftKindLabel}
+ </Select.Trigger>
+
+ <Select.Content class="font-mono text-xs">
+ {#each draftKinds as kind (kind)}
+ <Select.Item class="text-xs" label={kind.toUpperCase()} value={kind}>
+ {kind.toUpperCase()}
+ </Select.Item>
+ {/each}
+ </Select.Content>
+ </Select.Root>
+
+ <label class="cursor-pointer" for="discover-draft-sidecars">draft sidecar</label>
+ </div>
+ {/if}
+ </section>
+{/if}
+
+<DialogConfirmDownload
+ action={pending.action}
+ onClose={() => (confirmOpen = false)}
+ open={confirmOpen}
+ repoWithTag={pending.repoWithTag}
+/>
diff --git a/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsDownloadOptions/ModelsDiscoverDetailsDownloadOptionsDownloadCommand.svelte b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsDownloadOptions/ModelsDiscoverDetailsDownloadOptionsDownloadCommand.svelte
new file mode 100644
index 000000000..eb7f4e55f
--- /dev/null
+++ b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsDownloadOptions/ModelsDiscoverDetailsDownloadOptionsDownloadCommand.svelte
@@ -0,0 +1,131 @@
+<script lang="ts">
+ import { Check, Copy } from '@lucide/svelte';
+ import * as Select from '$lib/components/ui/select';
+ import { COPY_FEEDBACK_RESET_MS, MODEL_ID, SERVE_COMMAND, SPEC_TYPE } from '$lib/constants';
+ import type { ModelDraftSidecar } from '$lib/enums';
+ import { copyToClipboard } from '$lib/utils';
+
+ interface Props {
+ /** Full HuggingFace repo id, e.g. `ggml-org/gemma-3-4b-it-GGUF`. */
+ modelId: string;
+ /** Quant the command pins on `-hf`; null when the row's file carries none. */
+ quant?: string | null;
+ /**
+ * Draft kind the panel downloads with the model, null when the sidecars are
+ * off. The server resolves the draft file itself, so the command only names
+ * its type.
+ */
+ draftKind?: ModelDraftSidecar | null;
+ /** Draft kinds the repo ships; the command offers them in its own pick. */
+ draftKinds?: ModelDraftSidecar[];
+ /** Fires when the command's pick changes, so the panel's own pick follows. */
+ onDraftKindChange?: (kind: ModelDraftSidecar) => void;
+ }
+
+ let {
+ draftKind = null,
+ draftKinds = [],
+ modelId,
+ onDraftKindChange,
+ quant = null
+ }: Props = $props();
+
+ /** The llama serve command: the row's quant and, when it rides along, the draft. */
+ let command = $derived.by(() => {
+ const parts = [
+ SERVE_COMMAND.BIN,
+ SERVE_COMMAND.SUBCOMMAND,
+ SERVE_COMMAND.MODEL_FLAG,
+ quant ? `${modelId}${MODEL_ID.QUANTIZATION_SEPARATOR}${quant}` : modelId
+ ];
+
+ if (draftKind) {
+ parts.push(SERVE_COMMAND.SPEC_TYPE_FLAG, SPEC_TYPE[draftKind]);
+ }
+
+ return parts.join(' ');
+ });
+
+ let copied = $state(false);
+
+ async function copy() {
+ await copyToClipboard(command);
+ copied = true;
+ setTimeout(() => (copied = false), COPY_FEEDBACK_RESET_MS);
+ }
+</script>
+
+<div
+ class="relative flex items-center gap-2 overflow-hidden rounded-lg border border-border/40 bg-background py-2.5 pl-4 pr-10 shadow-xs dark:border-border/35 dark:bg-background/50"
+>
+ <!-- Single line: long commands scroll horizontally instead of wrapping. -->
+ <div
+ class="flex min-w-0 flex-1 items-center gap-x-2 overflow-x-auto py-0.5 font-mono text-xs whitespace-nowrap text-foreground/90"
+ >
+ <span class="shrink-0">{SERVE_COMMAND.BIN}</span>
+
+ <span class="shrink-0">{SERVE_COMMAND.SUBCOMMAND}</span>
+
+ <span class="shrink-0">{SERVE_COMMAND.MODEL_FLAG}</span>
+
+ <span class="shrink-0">
+ {modelId}{quant ? MODEL_ID.QUANTIZATION_SEPARATOR : ''}
+ </span>
+
+ <!-- the row's own quant, so no pick is needed here -->
+ {#if quant}
+ <span class="shrink-0 rounded-md border border-primary/15 bg-primary/[0.07] px-1.5">
+ {quant}
+ </span>
+ {/if}
+
+ <!-- The draft rides with the model, so the flag only names its type; the
+ checkbox below turns it on, the server resolves the file. The pick is
+ the panel's own, so changing it here changes it there. -->
+ {#if draftKind}
+ <span class="shrink-0">{SERVE_COMMAND.SPEC_TYPE_FLAG}</span>
+
+ <!-- the select only earns its chrome when there is a real choice to make -->
+ {#if draftKinds.length > 1}
+ <Select.Root
+ onValueChange={(value) => value && onDraftKindChange?.(value as ModelDraftSidecar)}
+ type="single"
+ value={draftKind}
+ >
+ <Select.Trigger
+ aria-label="Draft type"
+ class="border-primary/15 bg-primary/[0.07] font-mono text-foreground hover:bg-primary/15 focus-visible:border-primary/40 focus-visible:ring-0"
+ size="xs"
+ >
+ {SPEC_TYPE[draftKind]}
+ </Select.Trigger>
+
+ <Select.Content class="font-mono text-xs">
+ {#each draftKinds as kind (kind)}
+ <Select.Item class="text-xs" label={SPEC_TYPE[kind]} value={kind}>
+ {SPEC_TYPE[kind]}
+ </Select.Item>
+ {/each}
+ </Select.Content>
+ </Select.Root>
+ {:else}
+ <span class="shrink-0 rounded-md border border-primary/15 bg-primary/[0.07] px-1.5">
+ {SPEC_TYPE[draftKind]}
+ </span>
+ {/if}
+ {/if}
+ </div>
+
+ <button
+ aria-label="Copy command"
+ class="absolute top-1/2 right-2 -translate-y-1/2 cursor-pointer rounded-md p-1.5 text-muted-foreground/70 transition-colors hover:bg-primary/10 hover:text-foreground"
+ onclick={copy}
+ type="button"
+ >
+ {#if copied}
+ <Check class="h-3.5 w-3.5 text-green-500" />
+ {:else}
+ <Copy class="h-3.5 w-3.5" />
+ {/if}
+ </button>
+</div>
diff --git a/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsDownloadOptions/ModelsDiscoverDetailsDownloadOptionsDraftDownloadButton.svelte b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsDownloadOptions/ModelsDiscoverDetailsDownloadOptionsDraftDownloadButton.svelte
new file mode 100644
index 000000000..d7cbd8bec
--- /dev/null
+++ b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsDownloadOptions/ModelsDiscoverDetailsDownloadOptionsDraftDownloadButton.svelte
@@ -0,0 +1,61 @@
+<script lang="ts">
+ import { Download } from '@lucide/svelte';
+ import * as Tooltip from '$lib/components/ui/tooltip';
+ import { HuggingFaceService } from '$lib/services';
+ import { modelsStore } from '$lib/stores';
+ import type { ModelCompanionDraft } from '$lib/types';
+
+ interface Props {
+ /** The draft its model is missing, with its own file, quant and size. */
+ companion: ModelCompanionDraft;
+ /** Tag of the model the draft belongs to, the one the server pulls it with. */
+ repoWithTag: string;
+ }
+
+ let { companion, repoWithTag }: Props = $props();
+
+ // one source, so the chip matches the draft badge of a quant chip
+ const SIDECAR_BADGE_CLASS =
+ 'rounded-md bg-primary px-1 py-0.5 text-[10px] font-semibold tracking-wide text-primary-foreground uppercase';
+
+ let tooltipText = $derived(`Add the ${companion.kind} draft for ${repoWithTag}`);
+
+ // a draft is pulled with its model, so the model is posted again with the kind
+ function addDraft() {
+ void modelsStore.status.downloadModel(repoWithTag, [companion.kind]).catch(() => {
+ // the store already toasted the failure
+ });
+ }
+</script>
+
+<!-- A downloaded model missing its draft: the chip adds the draft alone, and
+ the model keeps the files it already has. -->
+<Tooltip.Root>
+ <Tooltip.Trigger>
+ {#snippet child({ props })}
+ <button
+ {...props}
+ aria-label={tooltipText}
+ class="group relative inline-flex h-auto cursor-pointer items-center gap-1 rounded-md border border-dashed border-border/60 px-2 py-1 text-left font-mono text-xs shadow-sm transition-[background-color,border-color,transform] duration-200 ease-[cubic-bezier(0.23,1,0.32,1)] hover:bg-muted-foreground/10 hover:border-border active:scale-[0.97] dark:border-border/40 dark:hover:bg-muted-foreground/15"
+ onclick={addDraft}
+ type="button"
+ >
+ <span class="{SIDECAR_BADGE_CLASS} inline-flex items-center gap-0.5">
+ {companion.kind}{companion.quant ? `:${companion.quant}` : ''}
+ </span>
+
+ <span class="-my-1 mx-0.75 w-px self-stretch bg-border"></span>
+
+ <span>{HuggingFaceService.formatFileSize(companion.size)}</span>
+
+ <Download
+ class="h-3.5 w-3.5 text-muted-foreground transition-colors duration-150 group-hover:text-foreground"
+ />
+ </button>
+ {/snippet}
+ </Tooltip.Trigger>
+
+ <Tooltip.Content>
+ <p>{tooltipText}</p>
+ </Tooltip.Content>
+</Tooltip.Root>
diff --git a/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsDownloadOptions/ModelsDiscoverDetailsDownloadOptionsQuantDownloadButton.svelte b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsDownloadOptions/ModelsDiscoverDetailsDownloadOptionsQuantDownloadButton.svelte
new file mode 100644
index 000000000..5ccbd31d0
--- /dev/null
+++ b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsDownloadOptions/ModelsDiscoverDetailsDownloadOptionsQuantDownloadButton.svelte
@@ -0,0 +1,272 @@
+<script lang="ts">
+ import ModelDownloadProgressBar from '../../../ModelDownloadProgressBar.svelte';
+ import { labelFor } from './download-options.utils';
+ import { Check, Download, Loader2, Pause, Play, RotateCw, X } from '@lucide/svelte';
+ import * as Tooltip from '$lib/components/ui/tooltip';
+ import { HuggingFaceService } from '$lib/services';
+ import { modelsStore } from '$lib/stores';
+ import type { ModelQuantChip } from '$lib/types';
+ import type { Snippet } from 'svelte';
+
+ interface Props {
+ /** The main quant chip: its file, its download state and its joined drafts. */
+ file: ModelQuantChip;
+ /** Ask the parent to confirm deleting a downloaded model; the chip owns no dialog. */
+ onRequestDelete?: (repoWithTag: string) => void;
+ /** Ask the parent to confirm cancelling an in-flight download. */
+ onRequestCancel?: (repoWithTag: string) => void;
+ }
+
+ let { file, onRequestCancel, onRequestDelete }: Props = $props();
+
+ let entry = $derived(file.state);
+
+ // one source, so the idle, in-flight and downloaded chip states stay identical
+ const SIDECAR_BADGE_CLASS =
+ 'rounded-md bg-primary px-1 py-0.5 text-[10px] font-semibold tracking-wide text-primary-foreground uppercase';
+
+ /** Queue the download; a failed attempt leaves partial files, drop them first. */
+ async function startDownload() {
+ try {
+ if (entry.isFailed) await modelsStore.status.cancelDownload(entry.repoWithTag);
+
+ // the joined drafts and the mmproj ride along: the server picks their files
+ await modelsStore.status.downloadModel(entry.repoWithTag, file.sidecars);
+ } catch {
+ // the store already toasted the failure
+ }
+ }
+
+ let label = $derived(labelFor(file.path));
+
+ let percent = $derived(
+ entry.progress && entry.progress.totalBytes > 0
+ ? Math.round((entry.progress.downloadedBytes / entry.progress.totalBytes) * 100)
+ : null
+ );
+
+ /** Draft the chip pulls with its quant, e.g. `MTP`; null when none matches. */
+ let companion = $derived(file.companion);
+
+ /**
+ * The joined draft reads once it rides along or lands: a downloaded model
+ * missing its draft keeps to its own weights, and the draft gets its own chip.
+ */
+ let showsCompanion = $derived(companion ? companion.onDisk || !entry.isDownloaded : false);
+
+ /** What the chip's actions name: the quant file and, when it rides along, its draft. */
+ let downloadLabel = $derived(
+ showsCompanion && companion ? `${file.path} + ${companion.path}` : file.path
+ );
+
+ let tooltipText = $derived(
+ entry.isDownloading
+ ? 'Pause downloading'
+ : entry.isPaused
+ ? 'Resume downloading'
+ : entry.isDownloaded
+ ? `Delete ${downloadLabel}`
+ : entry.isFailed
+ ? `Retry download: ${downloadLabel}`
+ : `Download ${downloadLabel}`
+ );
+
+ let downloadBytes = $derived(
+ (file.size ?? 0) + (showsCompanion && companion ? companion.size : 0)
+ );
+</script>
+
+{#snippet chipBody(dividerClass: string)}
+ <!-- label, joined draft and divider: identical on every chip state. The draft
+ rides with this quant, so it reads as part of the same chip:
+ `Q4_K_M + [MTP:Q8_0] | 20.0 GB`, the size covering both files. -->
+ <span class="font-medium">{label}</span>
+
+ {#if showsCompanion && companion}
+ <span class="text-muted-foreground/60">+</span>
+
+ <span class="{SIDECAR_BADGE_CLASS} inline-flex items-center gap-0.5">
+ <!-- the draft's own quant, so a 4-bit model paired with a 16-bit draft head reads at a glance -->
+ {companion.kind}{companion.quant ? `:${companion.quant}` : ''}
+
+ {#if companion.onDisk}
+ <Check class="h-2.5 w-2.5" />
+ {/if}
+ </span>
+ {/if}
+
+ <span class={dividerClass}></span>
+{/snippet}
+
+<!-- every chip is a tooltip trigger; the button renders as the trigger's child
+ so no nested button element is created -->
+{#snippet tooltipTrigger(tip: string, button: Snippet<[Record<string, unknown>]>)}
+ <Tooltip.Root>
+ <Tooltip.Trigger>
+ {#snippet child({ props })}
+ {@render button(props)}
+ {/snippet}
+ </Tooltip.Trigger>
+
+ <Tooltip.Content>
+ <p>{tip}</p>
+ </Tooltip.Content>
+ </Tooltip.Root>
+{/snippet}
+
+{#snippet deleteChip(props: Record<string, unknown>)}
+ <!-- downloaded chips are delete actions: green check by default, red X on hover -->
+ <button
+ {...props}
+ aria-label={tooltipText}
+ class="group relative inline-flex h-auto cursor-pointer items-center gap-1 rounded-md! border px-2 py-1 text-left font-mono text-xs shadow-sm transition-[background-color,border-color,transform] duration-200 ease-[cubic-bezier(0.23,1,0.32,1)] active:scale-[0.97]
+ border-green-600/25 bg-green-500/5 hover:border-destructive/50 hover:bg-destructive/10 dark:border-green-500/30 dark:bg-green-500/10 dark:hover:border-destructive/50 dark:hover:bg-destructive/15"
+ onclick={() => onRequestDelete?.(entry.repoWithTag)}
+ type="button"
+ >
+ {@render chipBody(
+ '-my-1 mx-0.75 w-px self-stretch bg-green-600/25 transition-colors duration-200 group-hover:bg-destructive/30 dark:bg-green-600/30 dark:group-hover:bg-destructive/30'
+ )}
+
+ <span>{HuggingFaceService.formatFileSize(downloadBytes)}</span>
+
+ <!-- icon slot: crossfade check -> x; touch devices show the delete affordance directly -->
+ <span class="relative inline-flex h-3.5 w-3.5 shrink-0 items-center justify-center">
+ <Check
+ class="absolute h-3.5 w-3.5 text-green-500 transition-[opacity,transform] duration-150 ease-[cubic-bezier(0.23,1,0.32,1)] group-hover:scale-75 group-hover:opacity-0 [@media(pointer:coarse)]:hidden"
+ />
+
+ <X
+ class="absolute h-3.5 w-3.5 scale-75 text-destructive opacity-0 transition-[opacity,transform] duration-150 ease-[cubic-bezier(0.23,1,0.32,1)] group-hover:scale-100 group-hover:opacity-100 [@media(pointer:coarse)]:scale-100 [@media(pointer:coarse)]:opacity-100"
+ />
+ </span>
+ </button>
+{/snippet}
+
+{#snippet pauseResumeChip(props: Record<string, unknown>)}
+ <button
+ {...props}
+ aria-label={tooltipText}
+ class="flex h-auto min-w-0 flex-1 cursor-pointer items-center gap-1 text-left"
+ onclick={() => {
+ if (entry.isDownloading) void modelsStore.status.pauseDownload(entry.repoWithTag);
+ else void modelsStore.status.downloadModel(entry.repoWithTag).catch(() => {});
+ }}
+ type="button"
+ >
+ {@render chipBody('-my-1 mx-0.75 w-px self-stretch bg-border')}
+
+ {#if percent !== null}
+ <span class="mr-1 tabular-nums">{percent}%</span>
+ {:else if entry.isPaused}
+ <span>Paused</span>
+ {:else}
+ <span>{HuggingFaceService.formatFileSize(downloadBytes)}</span>
+ {/if}
+
+ {#if entry.isDownloading}
+ <!-- spinner fades into the pause affordance on hover; opacity only, the
+ spin keyframes own the transform so scale would fight them -->
+ <span class="relative inline-flex h-3.5 w-3.5 shrink-0 items-center justify-center">
+ <Loader2
+ class="absolute h-3.5 w-3.5 animate-spin text-muted-foreground transition-opacity duration-150 ease-[cubic-bezier(0.23,1,0.32,1)] group-hover:opacity-0 [@media(pointer:coarse)]:hidden"
+ />
+
+ <Pause
+ class="absolute h-3.5 w-3.5 opacity-0 transition-opacity duration-150 ease-[cubic-bezier(0.23,1,0.32,1)] group-hover:opacity-100 [@media(pointer:coarse)]:opacity-100"
+ />
+ </span>
+ {:else}
+ <!-- paused: the state at rest, the resume affordance taking its place on
+ hover, the way the spinner gives way to the pause action above -->
+ <span class="relative inline-flex h-3.5 w-3.5 shrink-0 items-center justify-center">
+ <Pause
+ class="absolute h-3.5 w-3.5 text-muted-foreground transition-opacity duration-150 ease-[cubic-bezier(0.23,1,0.32,1)] group-hover:opacity-0 [@media(pointer:coarse)]:hidden"
+ />
+
+ <Play
+ class="absolute h-3.5 w-3.5 opacity-0 transition-opacity duration-150 ease-[cubic-bezier(0.23,1,0.32,1)] group-hover:opacity-100 [@media(pointer:coarse)]:opacity-100"
+ />
+ </span>
+ {/if}
+ </button>
+{/snippet}
+
+{#snippet cancelChip(props: Record<string, unknown>)}
+ <!-- cancel slot: same fixed slot and fade as the pause / play affordances
+ so the two icons line up exactly; the X turns destructive on its hover -->
+ <button
+ {...props}
+ aria-label="Cancel downloading"
+ class="relative grid h-3.5 w-3.5 shrink-0 cursor-pointer items-center justify-center text-muted-foreground/70"
+ onclick={() => onRequestCancel?.(entry.repoWithTag)}
+ type="button"
+ >
+ <X
+ class="h-3.5 w-3.5 transition-[opacity,transform,color] duration-150 ease-[cubic-bezier(0.23,1,0.32,1)] text-destructive [@media(pointer:coarse)]:scale-100 [@media(pointer:coarse)]:opacity-100"
+ />
+ </button>
+{/snippet}
+
+{#snippet downloadChip(props: Record<string, unknown>)}
+ <!-- idle chips download on click (retry when the last attempt failed) -->
+ <button
+ {...props}
+ aria-label={tooltipText}
+ class="group relative inline-flex h-auto cursor-pointer items-center gap-1 overflow-hidden rounded-md! border px-2 py-1 text-left font-mono text-xs shadow-sm transition-[background-color,border-color,transform] duration-200 ease-[cubic-bezier(0.23,1,0.32,1)] active:scale-[0.97]
+ border-border/30 bg-background hover:bg-muted-foreground/10 dark:border-border/20 dark:bg-muted-foreground/15 dark:text-secondary-foreground dark:hover:bg-muted-foreground/25
+ {entry.isFailed ? 'border-destructive!' : ''}"
+ onclick={() => void startDownload()}
+ type="button"
+ >
+ {#if entry.isFailed}
+ <span
+ class="rounded-md bg-destructive px-1 py-0.5 text-[10px] font-semibold tracking-wide text-destructive-foreground uppercase"
+ >
+ Failed
+ </span>
+ {/if}
+
+ {@render chipBody('-my-1 mx-0.75 w-px self-stretch bg-border')}
+
+ <span>{HuggingFaceService.formatFileSize(downloadBytes)}</span>
+
+ <span class="inline-flex h-3.5 w-3.5 shrink-0 items-center justify-center">
+ {#if entry.isFailed}
+ <RotateCw class="h-3.5 w-3.5 text-destructive" />
+ {:else}
+ <Download
+ class="h-3.5 w-3.5 text-muted-foreground transition-colors duration-150 group-hover:text-foreground"
+ />
+ {/if}
+ </span>
+ </button>
+{/snippet}
+
+{#if entry.isDownloaded}
+ {@render tooltipTrigger(tooltipText, deleteChip)}
+{:else if entry.isDownloading || entry.isPaused}
+ <!-- in-flight / paused chips: the chip body pauses / resumes on click, the X
+ inside the chip cancels the download. The X slot is reserved, so the chip
+ never reflows when the affordance fades in -->
+ <div
+ class="group relative inline-flex h-auto items-center gap-1 overflow-hidden rounded-md! border px-2 py-1 text-left font-mono text-xs shadow-sm transition-[background-color,border-color,transform] duration-200 ease-[cubic-bezier(0.23,1,0.32,1)] active:scale-[0.97]
+ {entry.isPaused
+ ? 'border-yellow-600/40 bg-yellow-500/10 hover:bg-yellow-500/20 dark:border-yellow-500/30 dark:bg-yellow-500/10'
+ : 'border-border/30 bg-background hover:bg-muted-foreground/10 dark:border-border/20 dark:bg-muted-foreground/15'}"
+ >
+ {@render tooltipTrigger(tooltipText, pauseResumeChip)}
+
+ {@render tooltipTrigger('Cancel downloading', cancelChip)}
+
+ {#if percent !== null}
+ <ModelDownloadProgressBar
+ downloadedBytes={entry.progress?.downloadedBytes ?? 0}
+ overlay
+ totalBytes={entry.progress?.totalBytes ?? 0}
+ />
+ {/if}
+ </div>
+{:else}
+ {@render tooltipTrigger(tooltipText, downloadChip)}
+{/if}
diff --git a/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsDownloadOptions/ModelsDiscoverDetailsDownloadOptionsRow.svelte b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsDownloadOptions/ModelsDiscoverDetailsDownloadOptionsRow.svelte
new file mode 100644
index 000000000..4527014fc
--- /dev/null
+++ b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsDownloadOptions/ModelsDiscoverDetailsDownloadOptionsRow.svelte
@@ -0,0 +1,134 @@
+<script lang="ts">
+ import ModelsDiscoverDetailsDownloadOptionsDownloadCommand from './ModelsDiscoverDetailsDownloadOptionsDownloadCommand.svelte';
+ import ModelsDiscoverDetailsDownloadOptionsDraftDownloadButton from './ModelsDiscoverDetailsDownloadOptionsDraftDownloadButton.svelte';
+ import ModelsDiscoverDetailsDownloadOptionsQuantDownloadButton from './ModelsDiscoverDetailsDownloadOptionsQuantDownloadButton.svelte';
+ import { ChevronDown } from '@lucide/svelte';
+ import {
+ BIT_DEPTH_LABEL_SUFFIX,
+ GIGABYTE_LABEL,
+ OTHER_BIT_DEPTH,
+ OTHER_BIT_DEPTH_LABEL
+ } from '$lib/constants';
+ import type { ModelDraftSidecar } from '$lib/enums';
+ import type { ModelQuantChip } from '$lib/types';
+ import { minMemoryTierGb } from '$lib/utils';
+ import { cubicOut } from 'svelte/easing';
+ import { slide } from 'svelte/transition';
+
+ interface Props {
+ /** Bit depth of the row; `99` renders as "Other". */
+ bitDepth: number;
+ /** The row's main quant chips. */
+ files: ModelQuantChip[];
+ /** Full HuggingFace repo id, for the row's command preview. */
+ modelId: string;
+ /** Draft kind the panel pulls with the model, null when the sidecars are off. */
+ draftKind?: ModelDraftSidecar | null;
+ /** Draft kinds the repo ships, for the command's own pick. */
+ draftKinds?: ModelDraftSidecar[];
+ /** Forwarded to the command: the panel follows its draft pick. */
+ onDraftKindChange?: (kind: ModelDraftSidecar) => void;
+ /** Forwarded to each chip: ask the parent to confirm a cancel. */
+ onRequestCancel?: (repoWithTag: string) => void;
+ /** Forwarded to each chip: ask the parent to confirm a delete. */
+ onRequestDelete?: (repoWithTag: string) => void;
+ }
+
+ let {
+ bitDepth,
+ draftKind = null,
+ draftKinds = [],
+ files,
+ modelId,
+ onDraftKindChange,
+ onRequestCancel,
+ onRequestDelete
+ }: Props = $props();
+
+ // collapsed until asked for; the panel remounts per model
+ let commandOpen = $state(false);
+
+ let mainFile = $derived(files[0] ?? null);
+ let mainMemGb = $derived(mainFile ? minMemoryTierGb(mainFile.size ?? 0) : null);
+ // the row's weights plus, when one rides along, the draft's own tier
+ let totalMemGb = $derived.by(() => {
+ if (mainMemGb === null) return null;
+
+ const draftGb = mainFile?.companion?.size ? minMemoryTierGb(mainFile.companion.size) : 0;
+
+ return mainMemGb + (draftGb ?? 0);
+ });
+
+ /** Drafts a downloaded quant is missing, each getting a chip of its own to add. */
+ let missingDrafts = $derived(
+ files.flatMap((file) =>
+ file.state.isDownloaded && file.companion && !file.companion.onDisk
+ ? [{ companion: file.companion, repoWithTag: file.state.repoWithTag }]
+ : []
+ )
+ );
+</script>
+
+<div class="py-3">
+ <div class="grid grid-cols-[5rem_1fr] items-center gap-3 @max-lg:grid-cols-1 @max-lg:gap-1.5">
+ <!-- the row's own label opens its command: it pins a quant of this row, so
+ the copy matches what the chips download -->
+ <button
+ aria-expanded={commandOpen}
+ class="flex w-fit cursor-pointer flex-col items-start gap-0.5 text-left text-sm tabular-nums text-muted-foreground transition-colors hover:text-foreground @max-lg:flex-row @max-lg:items-baseline @max-lg:gap-2"
+ onclick={() => (commandOpen = !commandOpen)}
+ type="button"
+ >
+ <span class="flex items-center gap-1">
+ {#if bitDepth === OTHER_BIT_DEPTH}
+ {OTHER_BIT_DEPTH_LABEL}
+ {:else}
+ {bitDepth}{BIT_DEPTH_LABEL_SUFFIX}
+ {/if}
+
+ <ChevronDown
+ class="h-3 w-3 shrink-0 transition-transform duration-200 ease-[cubic-bezier(0.23,1,0.32,1)] {commandOpen
+ ? 'rotate-180'
+ : ''}"
+ />
+ </span>
+
+ {#if totalMemGb}
+ <span
+ class="text-[10px] whitespace-nowrap text-muted-foreground/60 @max-lg:whitespace-normal"
+ >
+ needs at least {totalMemGb}{GIGABYTE_LABEL} memory
+ </span>
+ {/if}
+ </button>
+
+ <div class="flex flex-wrap justify-end gap-1.5 @max-lg:justify-start">
+ {#each files as file (file.path)}
+ <ModelsDiscoverDetailsDownloadOptionsQuantDownloadButton
+ {file}
+ {onRequestCancel}
+ {onRequestDelete}
+ />
+ {/each}
+
+ {#each missingDrafts as draft (draft.repoWithTag + ':' + draft.companion.kind)}
+ <ModelsDiscoverDetailsDownloadOptionsDraftDownloadButton
+ companion={draft.companion}
+ repoWithTag={draft.repoWithTag}
+ />
+ {/each}
+ </div>
+ </div>
+
+ {#if commandOpen}
+ <div transition:slide={{ duration: 200, easing: cubicOut }} class="pt-2">
+ <ModelsDiscoverDetailsDownloadOptionsDownloadCommand
+ {draftKind}
+ {draftKinds}
+ {modelId}
+ {onDraftKindChange}
+ quant={mainFile?.quant ?? null}
+ />
+ </div>
+ {/if}
+</div>
diff --git a/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsDownloadOptions/download-options.utils.ts b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsDownloadOptions/download-options.utils.ts
new file mode 100644
index 000000000..3c07cbafd
--- /dev/null
+++ b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsDownloadOptions/download-options.utils.ts
@@ -0,0 +1,57 @@
+import { DRAFT_FILE_LABEL, MODEL_ID, PATH_SEPARATOR } from '$lib/constants';
+import { ModelDraftSidecar, ModelSelectableFileKind } from '$lib/enums';
+import { HuggingFaceService } from '$lib/services';
+import type { ModelSelectableFile } from '$lib/types';
+import { isAuxSidecar, isDraftSidecar } from '$lib/utils';
+
+/**
+ * Draft kinds in the order a chip lists them. DFlash wins: on the models that
+ * ship both it drafts a block at a time and stays accurate further out than
+ * MTP's few tokens; a kind not listed here sorts last.
+ */
+export const DRAFT_PREFERENCE_ORDER: readonly ModelDraftSidecar[] = [
+ ModelDraftSidecar.DFLASH,
+ ModelDraftSidecar.MTP
+];
+
+/** Draft sidecar of a file, typed for files already known to be drafts; null when it carries none. */
+export function draftBadgeOf(file: Pick<ModelSelectableFile, 'path'>): ModelDraftSidecar | null {
+ const sidecar = HuggingFaceService.extractQuantMeta(file.path)?.sidecar;
+
+ return sidecar && isDraftSidecar(sidecar) ? sidecar : null;
+}
+
+/** Kind of a file path: the main weights, a draft sidecar, or an aux sidecar (mmproj). */
+export function classify(path: string): ModelSelectableFileKind {
+ const sidecar = HuggingFaceService.extractQuantMeta(path)?.sidecar;
+
+ if (!sidecar) return ModelSelectableFileKind.MAIN;
+
+ return isAuxSidecar(sidecar) ? ModelSelectableFileKind.AUX : ModelSelectableFileKind.DRAFT;
+}
+
+/** Draft kinds best first, so a chip lists its joined badges in the pull order. */
+export function sortDraftKinds(kinds: readonly ModelDraftSidecar[]): ModelDraftSidecar[] {
+ const rank = (kind: ModelDraftSidecar) => {
+ const index = DRAFT_PREFERENCE_ORDER.indexOf(kind);
+
+ return index === -1 ? DRAFT_PREFERENCE_ORDER.length : index;
+ };
+
+ return [...kinds].sort((a, b) => rank(a) - rank(b));
+}
+
+/** Display label of a file: its quant, else the file name without the extension. */
+export function labelFor(path: string): string {
+ const meta = HuggingFaceService.extractQuantMeta(path);
+
+ if (meta?.quant) return meta.quant;
+
+ // Quantless sidecar files: the chip badge already carries the sidecar type,
+ // so the label only marks draft files; aux sidecars badge alone.
+ if (meta?.sidecar) return isDraftSidecar(meta.sidecar) ? DRAFT_FILE_LABEL : '';
+
+ const basename = path.split(PATH_SEPARATOR).pop() ?? path;
+
+ return basename.replace(MODEL_ID.WEIGHT_EXTENSION_REGEX, '');
+}
diff --git a/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsDownloadOptions/index.ts b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsDownloadOptions/index.ts
new file mode 100644
index 000000000..bb82218cc
--- /dev/null
+++ b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsDownloadOptions/index.ts
@@ -0,0 +1,24 @@
+/**
+ *
+ * MODELS DISCOVER - DETAILS - DOWNLOAD OPTIONS
+ *
+ * The download area of the detail pane: one chip per main quant, the drafts it rides with
+ * badged on it, a draft checkbox and type pick at the bottom, and a terminal command
+ * preview inside every row.
+ *
+ */
+
+/** One download action per main quant, the sidecars it pulls, and the row's command. */
+export { default as ModelsDiscoverDetailsDownloadOptions } from './ModelsDiscoverDetailsDownloadOptions.svelte';
+
+/** One bit-depth row: the depth label opens its command, the quants of that depth download beside it. */
+export { default as ModelsDiscoverDetailsDownloadOptionsRow } from './ModelsDiscoverDetailsDownloadOptionsRow.svelte';
+
+/** A single GGUF file as an independent action chip. */
+export { default as ModelsDiscoverDetailsDownloadOptionsQuantDownloadButton } from './ModelsDiscoverDetailsDownloadOptionsQuantDownloadButton.svelte';
+
+/** Chip for a draft a downloaded model is missing; posts the model with that kind, so the server keeps what is on disk and pulls the draft. */
+export { default as ModelsDiscoverDetailsDownloadOptionsDraftDownloadButton } from './ModelsDiscoverDetailsDownloadOptionsDraftDownloadButton.svelte';
+
+/** The `llama serve -hf ...` command box of one row, with a draft type pick and a copy button. */
+export { default as ModelsDiscoverDetailsDownloadOptionsDownloadCommand } from './ModelsDiscoverDetailsDownloadOptionsDownloadCommand.svelte';
diff --git a/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsHeader.svelte b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsHeader.svelte
new file mode 100644
index 000000000..868765079
--- /dev/null
+++ b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsHeader.svelte
@@ -0,0 +1,96 @@
+<script lang="ts">
+ import ModelCapabilityIcons from '../../ModelCapabilityIcons.svelte';
+ import ModelOrgAvatar from '../../ModelOrgAvatar.svelte';
+ import ModelsDiscoverDetailsHfHubStats from './ModelsDiscoverDetailsHfHubStats.svelte';
+ import ModelsDiscoverDetailsMetadata from './ModelsDiscoverDetailsMetadata.svelte';
+ import { ExternalLink } from '@lucide/svelte';
+ import { base } from '$app/paths';
+ import { Button } from '$lib/components/ui/button';
+ import { HuggingFaceService } from '$lib/services';
+ import type { HfModelDetailInfo, HfModelGguf } from '$lib/types/huggingface';
+ import { orgOf } from '$lib/utils';
+
+ interface Props {
+ modelId: string;
+ details: HfModelDetailInfo;
+ gguf?: HfModelGguf;
+ baseModels: string[];
+ licenseTag: string | null;
+ hasVision: boolean;
+ hasTools: boolean;
+ hasReasoning: boolean;
+ }
+
+ let { baseModels, details, gguf, hasReasoning, hasTools, hasVision, licenseTag, modelId }: Props =
+ $props();
+
+ // Avatar shows the base model's org (e.g. the Qwen logo for a ggml-org GGUF)
+ // with the quant org as a corner badge when they differ.
+ let repoOrg = $derived(orgOf(details.id) || orgOf(modelId));
+ let baseOrg = $derived(orgOf(baseModels[0]));
+ let avatarOrg = $derived(baseOrg || repoOrg);
+ let quantOrg = $derived(baseOrg && baseOrg !== repoOrg ? repoOrg : undefined);
+</script>
+
+<header class="space-y-3">
+ <div class="flex flex-wrap items-start justify-between gap-3">
+ <div class="flex min-w-0 items-center gap-2">
+ <ModelOrgAvatar
+ org={avatarOrg}
+ {quantOrg}
+ quantPositionClass="-bottom-1.5 -right-1.5"
+ quantSize="h-6 w-6"
+ size="h-12 w-12"
+ />
+
+ <div class="min-w-0">
+ <div class="flex items-center gap-2 @max-lg:flex-col @max-lg:items-start">
+ <h1 class="truncate text-lg font-semibold @max-lg:w-full">{details.id ?? modelId}</h1>
+
+ <ModelCapabilityIcons
+ gapClass="gap-2"
+ iconSize="h-4 w-4"
+ modalities={{ audio: false, video: false, vision: hasVision }}
+ supportsThinking={hasReasoning}
+ supportsToolUse={hasTools}
+ />
+ </div>
+
+ {#if baseModels.length}
+ <div class="flex items-center gap-1">
+ <span class="truncate text-xs text-muted-foreground">{baseModels.join(', ')}</span>
+
+ <a
+ aria-label="View base model on HuggingFace"
+ class="shrink-0 text-muted-foreground transition-colors hover:text-foreground"
+ href={HuggingFaceService.getModelUrl(baseModels[0])}
+ rel="noopener noreferrer"
+ target="_blank"
+ >
+ <ExternalLink class="h-3 w-3" />
+ </a>
+ </div>
+ {/if}
+ </div>
+ </div>
+
+ <Button
+ class="h-auto gap-1.5 px-2.5 py-1.5 text-xs"
+ href={HuggingFaceService.getModelUrl(modelId)}
+ rel="noopener noreferrer"
+ size="sm"
+ target="_blank"
+ variant="tertiary"
+ >
+ <img alt="" class="h-3.5 w-3.5" src="{base}/recommended-mcp/huggingface.ico" />
+
+ View on Hugging Face
+
+ <ExternalLink class="size-3.5" />
+ </Button>
+ </div>
+
+ <ModelsDiscoverDetailsHfHubStats {details} />
+
+ <ModelsDiscoverDetailsMetadata {details} {gguf} {licenseTag} {modelId} />
+</header>
diff --git a/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsHfHubStats.svelte b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsHfHubStats.svelte
new file mode 100644
index 000000000..b848a9bfe
--- /dev/null
+++ b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsHfHubStats.svelte
@@ -0,0 +1,31 @@
+<script lang="ts">
+ import { Download, Heart } from '@lucide/svelte';
+ import { HuggingFaceService } from '$lib/services';
+ import type { HfModelDetailInfo } from '$lib/types/huggingface';
+
+ interface Props {
+ details: HfModelDetailInfo;
+ }
+
+ let { details }: Props = $props();
+</script>
+
+<div class="flex flex-wrap items-center gap-x-4 gap-y-1 text-sm text-muted-foreground">
+ {#if typeof details.downloads === 'number'}
+ <span class="inline-flex items-center gap-1.5">
+ <Download class="h-3.5 w-3.5" />
+ {HuggingFaceService.formatDownloads(details.downloads)}
+ </span>
+ {/if}
+
+ {#if typeof details.likes === 'number'}
+ <span class="inline-flex items-center gap-1.5">
+ <Heart class="h-3.5 w-3.5" />
+ {HuggingFaceService.formatLikes(details.likes)}
+ </span>
+ {/if}
+
+ {#if details.lastModified}
+ <span>Updated {HuggingFaceService.formatRelativeTime(details.lastModified)}</span>
+ {/if}
+</div>
diff --git a/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsMetadata.svelte b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsMetadata.svelte
new file mode 100644
index 000000000..af7a38089
--- /dev/null
+++ b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsMetadata.svelte
@@ -0,0 +1,82 @@
+<script lang="ts">
+ import ModelsDiscoverChatTemplateDialog from './ModelsDiscoverChatTemplateDialog.svelte';
+ import ModelsDiscoverDetailsMetadataItem from './ModelsDiscoverDetailsMetadataItem.svelte';
+ import { MessageSquareCode } from '@lucide/svelte';
+ import { Button } from '$lib/components/ui/button';
+ import { modelsDiscoverStore } from '$lib/stores';
+ import type { HfModelDetailInfo, HfModelGguf } from '$lib/types/huggingface';
+ import { formatParameters } from '$lib/utils';
+
+ interface Props {
+ /** Full HuggingFace model id, e.g. `ggml-org/gemma-3-4b-it-GGUF`. */
+ modelId: string;
+ details: HfModelDetailInfo;
+ gguf?: HfModelGguf;
+ licenseTag: string | null;
+ }
+
+ let { details, gguf, licenseTag, modelId }: Props = $props();
+
+ // Catalog family description when curated, else the HF card description.
+ let description = $derived(
+ modelsDiscoverStore.descriptionFor(modelId) ?? details.cardData?.description
+ );
+
+ let chatTemplateOpen = $state(false);
+</script>
+
+{#if description}
+ <p class="text-sm text-muted-foreground">{description}</p>
+{/if}
+
+<!-- Metadata chips: label | value pairs, matching the HF model page style -->
+<div class="flex flex-wrap items-center gap-1.5">
+ {#if gguf?.total}
+ <ModelsDiscoverDetailsMetadataItem
+ label="Model size"
+ value="{formatParameters(gguf.total)} params"
+ />
+ {/if}
+
+ {#if gguf?.context_length}
+ <ModelsDiscoverDetailsMetadataItem
+ label="Context"
+ value={gguf.context_length.toLocaleString()}
+ />
+ {/if}
+
+ {#if gguf?.architecture}
+ <ModelsDiscoverDetailsMetadataItem label="Architecture" value={gguf.architecture} />
+ {/if}
+
+ {#if gguf?.chat_template}
+ <Button
+ class="h-auto gap-1.5 px-2.5 py-1 text-xs"
+ onclick={() => (chatTemplateOpen = true)}
+ size="sm"
+ variant="tertiary"
+ >
+ <MessageSquareCode class="size-3" />
+ Chat template
+ </Button>
+ {/if}
+
+ {#if licenseTag}
+ <ModelsDiscoverDetailsMetadataItem label="License" value={licenseTag} />
+ {/if}
+
+ {#if details.gated === true}
+ <span
+ class="rounded bg-yellow-500/10 px-2 py-0.5 text-xs font-medium text-yellow-600 dark:text-yellow-400"
+ >
+ gated
+ </span>
+ {/if}
+</div>
+
+{#if gguf?.chat_template}
+ <ModelsDiscoverChatTemplateDialog
+ bind:open={chatTemplateOpen}
+ chatTemplate={gguf.chat_template}
+ />
+{/if}
diff --git a/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsMetadataItem.svelte b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsMetadataItem.svelte
new file mode 100644
index 000000000..4bfc027d8
--- /dev/null
+++ b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsMetadataItem.svelte
@@ -0,0 +1,18 @@
+<script lang="ts">
+ interface Props {
+ label: string;
+ value: string;
+ }
+
+ let { label, value }: Props = $props();
+</script>
+
+<!-- Metadata chip: label | value pair, matching the HF model page style, on the
+ raised surface the tertiary button variant uses -->
+<span
+ class="inline-flex items-center divide-x divide-border rounded-md border border-border/30 bg-muted/60 text-xs shadow-sm dark:border-border/20 dark:bg-muted/75"
+>
+ <span class="px-2.5 py-1 text-muted-foreground">{label}</span>
+
+ <span class="px-2.5 py-1 font-medium">{value}</span>
+</span>
diff --git a/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsReadme.svelte b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsReadme.svelte
new file mode 100644
index 000000000..fbf043b79
--- /dev/null
+++ b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsReadme.svelte
@@ -0,0 +1,17 @@
+<script lang="ts">
+ import { MarkdownContent } from '$lib/components/app';
+
+ interface Props {
+ readme: string | null;
+ }
+
+ let { readme }: Props = $props();
+</script>
+
+{#if readme}
+ <section
+ class="p-3 rounded-3xl border border-border/30 bg-muted/60 shadow-sm dark:border-border/20"
+ >
+ <MarkdownContent allowHtml class="prose-sm max-w-none" content={readme} />
+ </section>
+{/if}
diff --git a/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsSkeleton.svelte b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsSkeleton.svelte
new file mode 100644
index 000000000..8438a3c2b
--- /dev/null
+++ b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/ModelsDiscoverDetailsSkeleton.svelte
@@ -0,0 +1,98 @@
+<script lang="ts">
+ import { Skeleton } from '$lib/components/ui/skeleton';
+</script>
+
+<!-- Static skeleton of ModelsDiscoverDetails: header, metadata chips, download options and readme lines. -->
+<div class="@container space-y-6 p-6 pr-0" data-slot="model-details-skeleton">
+ <!-- Header: avatar, name with capability icons, base model line, HF link -->
+ <div class="space-y-3">
+ <div class="flex items-start justify-between gap-3">
+ <div class="flex min-w-0 items-center gap-2">
+ <Skeleton class="h-12 w-12 rounded-md" />
+
+ <div class="min-w-0 space-y-1.5">
+ <Skeleton class="h-5 w-56 max-w-full" />
+
+ <Skeleton class="h-3 w-40 max-w-full" />
+ </div>
+ </div>
+
+ <Skeleton class="h-7.5 w-44 shrink-0 rounded-md" />
+ </div>
+
+ <!-- downloads / likes / last updated -->
+ <div class="flex items-center gap-4">
+ <Skeleton class="h-3.5 w-16" />
+
+ <Skeleton class="h-3.5 w-12" />
+
+ <Skeleton class="h-3.5 w-24" />
+ </div>
+
+ <!-- catalog family description, else the HF card description -->
+ <Skeleton class="h-3.5 my-4 w-full" />
+
+ <!-- metadata chips -->
+ <div class="space-y-3">
+ <div class="flex flex-wrap items-center gap-1.5">
+ <Skeleton class="h-7 w-36 rounded-md" />
+
+ <Skeleton class="h-7 w-28 rounded-md" />
+
+ <Skeleton class="h-7 w-32 rounded-md" />
+
+ <Skeleton class="h-7 w-24 rounded-md" />
+ </div>
+ </div>
+ </div>
+
+ <!-- Download options: quant rows and the draft opt-in line -->
+ <div class="rounded-3xl border border-border/30 bg-muted/60 shadow-sm dark:border-border/20">
+ <div class="flex w-full flex-col divide-y divide-border/50 px-4 pb-1 dark:divide-border/35">
+ {#each [0, 1, 2] as _, index (index)}
+ <div
+ class="grid grid-cols-[5rem_1fr] items-center gap-3 py-3 @max-lg:grid-cols-1 @max-lg:gap-1.5"
+ >
+ <div class="space-y-1.5">
+ <Skeleton class="h-4 w-14" />
+
+ <Skeleton class="h-2.5 w-24" />
+ </div>
+
+ <div class="flex flex-wrap items-center justify-end gap-1.5">
+ <Skeleton class="h-6 w-28 rounded-md" />
+
+ <Skeleton class="h-6 w-20 rounded-md" />
+ </div>
+ </div>
+ {/each}
+ </div>
+
+ <div class="space-y-2.5 border-t border-border/50 px-4 pt-3.5 dark:border-border/35">
+ <Skeleton class="h-9 w-full rounded-md" />
+
+ <div class="flex items-center gap-3">
+ <span class="h-px flex-1 bg-border/50"></span>
+
+ <Skeleton class="h-3 w-44" />
+
+ <span class="h-px flex-1 bg-border/50"></span>
+ </div>
+
+ <Skeleton class="h-11 w-full rounded-md" />
+ </div>
+ </div>
+
+ <!-- Readme -->
+ <div class="p-3 rounded-3xl border border-border/30 bg-muted/60 shadow-sm dark:border-border/20">
+ <div class="space-y-2.5">
+ <Skeleton class="h-3.5 w-full" />
+
+ <Skeleton class="h-3.5 w-full" />
+
+ <Skeleton class="h-3.5 w-5/6" />
+
+ <Skeleton class="h-3.5 w-2/3" />
+ </div>
+ </div>
+</div>
diff --git a/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/index.ts b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/index.ts
new file mode 100644
index 000000000..91d5c8906
--- /dev/null
+++ b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverDetails/index.ts
@@ -0,0 +1,37 @@
+/**
+ *
+ * MODELS DISCOVER - DETAILS
+ *
+ * The right-hand detail pane of the discover view: header (avatar, name, stats, metadata
+ * chips, capability badges), the download options area and the model-card README.
+ *
+ */
+
+/** Detail pane of the selected model; fetches its own details and GGUF file list. */
+export { default as ModelsDiscoverDetails } from './ModelsDiscoverDetails.svelte';
+
+/** Fetches one repo's details, files and readme for hosts outside Discover. */
+export { default as ModelsDiscoverDetailsContainer } from './ModelsDiscoverDetailsContainer.svelte';
+
+/** Header of the detail pane. */
+export { default as ModelsDiscoverDetailsHeader } from './ModelsDiscoverDetailsHeader.svelte';
+
+/** Downloads, likes and last updated, formatted the way the Hub shows them. */
+export { default as ModelsDiscoverDetailsHfHubStats } from './ModelsDiscoverDetailsHfHubStats.svelte';
+
+/** Static placeholder matching the detail layout. */
+export { default as ModelsDiscoverDetailsSkeleton } from './ModelsDiscoverDetailsSkeleton.svelte';
+
+/** Model card README as markdown. */
+export { default as ModelsDiscoverDetailsReadme } from './ModelsDiscoverDetailsReadme.svelte';
+
+/** Metadata chips: size, context, architecture, license, chat template. */
+export { default as ModelsDiscoverDetailsMetadata } from './ModelsDiscoverDetailsMetadata.svelte';
+
+/** One label | value chip of the metadata row. */
+export { default as ModelsDiscoverDetailsMetadataItem } from './ModelsDiscoverDetailsMetadataItem.svelte';
+
+/** Chat template in a scrollable dialog with a copy button. */
+export { default as ModelsDiscoverChatTemplateDialog } from './ModelsDiscoverChatTemplateDialog.svelte';
+
+export * from './ModelsDiscoverDetailsDownloadOptions';
diff --git a/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverList/ModelsDiscoverList.svelte b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverList/ModelsDiscoverList.svelte
new file mode 100644
index 000000000..5ebff7230
--- /dev/null
+++ b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverList/ModelsDiscoverList.svelte
@@ -0,0 +1,41 @@
+<script lang="ts">
+ import ModelsDiscoverListItem from './ModelsDiscoverListItem.svelte';
+ import ModelsDiscoverListItemSkeleton from './ModelsDiscoverListItemSkeleton.svelte';
+ import type { HfModelInfo } from '$lib/types/huggingface';
+
+ interface Props {
+ models: HfModelInfo[];
+ activeId?: string | null;
+ loading?: boolean;
+ loadingSkeletonRowsCount?: number;
+ /** Show the original (base) model's org avatar instead of the repo's org. */
+ showBaseModelAvatar?: boolean;
+ onSelect?: (modelId: string) => void;
+ }
+
+ let {
+ activeId = null,
+ loading = false,
+ loadingSkeletonRowsCount = 8,
+ models,
+ onSelect,
+ showBaseModelAvatar = false
+ }: Props = $props();
+</script>
+
+<ul class="space-y-0.5">
+ {#if loading}
+ {#each Array(loadingSkeletonRowsCount) as _, index (index)}
+ <ModelsDiscoverListItemSkeleton {index} />
+ {/each}
+ {:else}
+ {#each models as model (model.id)}
+ <ModelsDiscoverListItem
+ active={model.id === activeId}
+ {model}
+ {onSelect}
+ {showBaseModelAvatar}
+ />
+ {/each}
+ {/if}
+</ul>
diff --git a/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverList/ModelsDiscoverListItem.svelte b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverList/ModelsDiscoverListItem.svelte
new file mode 100644
index 000000000..1eee85ea0
--- /dev/null
+++ b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverList/ModelsDiscoverListItem.svelte
@@ -0,0 +1,256 @@
+<script lang="ts">
+ import ModelDownloadProgressBar from '../../ModelDownloadProgressBar.svelte';
+ import ModelDraftSubline from '../../ModelDraftSubline.svelte';
+ import ModelId from '../../ModelId.svelte';
+ import ModelOrgAvatar from '../../ModelOrgAvatar.svelte';
+ import { Pause, Play, X } from '@lucide/svelte';
+ import { ActionIcon } from '$lib/components/app';
+ import { HF_MMPROJ_FILENAME_TOKEN, HF_MODALITY_PIPELINE_TAGS, MODEL_ID } from '$lib/constants';
+ import { ModelDraftSidecar, ModelRowDownloadState } from '$lib/enums';
+ import { HuggingFaceService } from '$lib/services';
+ import { modelsDiscoverStore, modelsStore } from '$lib/stores';
+ import type { ModelsDiscoverSizeRange } from '$lib/stores/models-discover/index.svelte';
+ import type { HfModelInfo, HfModelSiblingRef } from '$lib/types/huggingface';
+ import type {
+ ModelDownloadProgress,
+ ModelModalities,
+ ModelSidecarBadge
+ } from '$lib/types/models';
+ import {
+ detectThinkingSupport,
+ detectToolUseSupport,
+ directoryOf,
+ isDraftSidecar,
+ isServerSidecarFile,
+ orgOf
+ } from '$lib/utils';
+ import { SvelteSet } from 'svelte/reactivity';
+
+ /**
+ * True when a file of the weights sits in the draft's directory: the server
+ * resolves a sidecar against the model file beside it, never across directories.
+ */
+ function hasWeightsBeside(siblings: HfModelSiblingRef[], draftPath: string): boolean {
+ const directory = directoryOf(draftPath);
+
+ return siblings.some(
+ (sibling) =>
+ MODEL_ID.WEIGHT_EXTENSION_REGEX.test(sibling.rfilename) &&
+ directoryOf(sibling.rfilename) === directory &&
+ !HuggingFaceService.extractQuantMeta(sibling.rfilename)?.sidecar
+ );
+ }
+
+ interface Props {
+ model: HfModelInfo;
+ active?: boolean;
+ /** State of a tracked download this row stands for, else the row is a plain result. */
+ download?: {
+ isPaused: boolean;
+ /** Live while the download runs, frozen while it is paused. */
+ progress: ModelDownloadProgress | null;
+ repoWithTag: string;
+ /** Ask the list to confirm cancelling; the list owns the single dialog. */
+ onRequestCancel?: (repoWithTag: string) => void;
+ };
+ /** Show the original (base) model's org avatar instead of the repo's org. */
+ showBaseModelAvatar?: boolean;
+ /** Draft sidecars already on disk, rendered as checked sub-lines. */
+ sidecarBadges?: ModelSidecarBadge[];
+ onSelect?: (modelId: string) => void;
+ }
+
+ let {
+ active = false,
+ download,
+ model,
+ onSelect,
+ showBaseModelAvatar = false,
+ sidecarBadges = []
+ }: Props = $props();
+
+ /** A draft still coming down badges the id, an on-disk one keeps its sub-line. */
+ let pendingDrafts = $derived(sidecarBadges.filter((badge) => badge.pending));
+ let onDiskDrafts = $derived(sidecarBadges.filter((badge) => !badge.pending));
+
+ let percent = $derived(
+ download?.progress && download.progress.totalBytes > 0
+ ? Math.round((download.progress.downloadedBytes / download.progress.totalBytes) * 100)
+ : null
+ );
+
+ let org = $derived(orgOf(model.id));
+
+ // Org whose avatar is shown: the base model's org when showBaseModelAvatar
+ // (e.g. the Qwen logo for ggml-org/Qwen3.8-27B-GGUF), else the repo's org.
+ let avatarOrg = $derived.by(() => {
+ if (!showBaseModelAvatar) return org;
+
+ return orgOf(HuggingFaceService.getBaseModels(model)[0]) || org;
+ });
+
+ let contextLength = $derived(model.gguf?.context_length);
+
+ // Reasoning support from the chat template, matching the details view.
+ let supportsThinking = $derived(detectThinkingSupport(model.gguf?.chat_template ?? ''));
+
+ let supportsToolUse = $derived(detectToolUseSupport(model.gguf?.chat_template ?? ''));
+
+ // Modalities derived from HF metadata: vision from an mmproj sidecar or a
+ // multimodal pipeline tag, audio/video from their pipeline tags.
+ let modalities = $derived.by<ModelModalities>(() => {
+ const tag = model.pipeline_tag ?? '';
+ const vision =
+ HF_MODALITY_PIPELINE_TAGS.vision.includes(tag) ||
+ Boolean(
+ model.siblings?.some((s) => s.rfilename.toLowerCase().includes(HF_MMPROJ_FILENAME_TOKEN))
+ );
+ const audio = HF_MODALITY_PIPELINE_TAGS.audio.includes(tag);
+ const video = HF_MODALITY_PIPELINE_TAGS.video.includes(tag);
+
+ return { audio, video, vision };
+ });
+
+ // Draft sidecars (mtp, dflash, dspark, eagle3) the repo offers, badged with no
+ // quant: the list does not know how the server configures them. Only the drafts
+ // the server can actually pull are shown, so a badge never outlives the file
+ // find_best_sibling would resolve.
+ let draftKinds = $derived.by<ModelDraftSidecar[]>(() => {
+ const siblings = model.siblings ?? [];
+ const kinds = new SvelteSet<ModelDraftSidecar>();
+
+ for (const sibling of siblings) {
+ const sidecar = HuggingFaceService.extractQuantMeta(sibling.rfilename)?.sidecar;
+
+ if (!sidecar || !isDraftSidecar(sidecar)) continue;
+
+ if (!isServerSidecarFile(sibling.rfilename, sidecar)) continue;
+
+ if (!hasWeightsBeside(siblings, sibling.rfilename)) continue;
+
+ kinds.add(sidecar);
+ }
+
+ return Object.values(ModelDraftSidecar).filter((kind) => kinds.has(kind));
+ });
+
+ // Min/max size across the repo's quants, draft sidecars included. The store
+ // has catalog rows covered already; any other row (a search hit) measures
+ // its repo once here and the result is cached per repo.
+ let measuredSize = $state<ModelsDiscoverSizeRange | null>(null);
+ let sizeRange = $derived(modelsDiscoverStore.cachedSizeRangeFor(model.id) ?? measuredSize);
+
+ $effect(() => {
+ const id = model.id;
+
+ if (modelsDiscoverStore.cachedSizeRangeFor(id)) return;
+
+ let cancelled = false;
+
+ void modelsDiscoverStore.sizeRange(id).then((range) => {
+ if (!cancelled) measuredSize = range ?? null;
+ });
+
+ return () => {
+ cancelled = true;
+ };
+ });
+</script>
+
+<li
+ class="group relative overflow-hidden rounded-lg transition-colors {active
+ ? 'bg-primary/10 hover:bg-primary/15'
+ : 'hover:bg-muted/60'}"
+>
+ <div class="flex items-center gap-0.5">
+ <button
+ aria-current={active ? 'page' : undefined}
+ class="flex min-w-0 flex-1 cursor-pointer items-start gap-2.5 p-2.5 text-left"
+ onclick={() => onSelect?.(model.id)}
+ type="button"
+ >
+ <ModelOrgAvatar
+ class="mt-1"
+ org={avatarOrg}
+ quantOrg={showBaseModelAvatar ? org : undefined}
+ />
+
+ <span class="min-w-0 flex-1">
+ <!-- a download row names what is downloading: the tag's quant and sidecar
+ badges, not the repo's whole sidecar offering -->
+ <ModelId
+ class="min-w-0"
+ {contextLength}
+ draftKinds={download ? undefined : draftKinds}
+ draftSidecars={download ? pendingDrafts : undefined}
+ hideOrgName
+ iconsOnNewLine
+ {modalities}
+ modelId={download ? download.repoWithTag : model.id}
+ {sizeRange}
+ {supportsThinking}
+ {supportsToolUse}
+ wrap
+ />
+ </span>
+ </button>
+
+ {#if download}
+ {@const state = download}
+
+ <!-- the percent sits on the row's centre line, and the download's controls take
+ its place on hover, the way the table's status column does -->
+ <span class="flex shrink-0 items-center gap-0.5 pr-2.5">
+ <span
+ class="text-xs text-muted-foreground tabular-nums group-hover:hidden [@media(pointer:coarse)]:hidden"
+ >
+ {percent !== null ? `${percent}%` : state.isPaused ? 'Paused' : 'Downloading'}
+ </span>
+
+ <span class="hidden items-center gap-0.5 group-hover:flex [@media(pointer:coarse)]:flex">
+ <ActionIcon
+ icon={state.isPaused ? Play : Pause}
+ iconSize="h-4 w-4"
+ onclick={() => void modelsStore.status.toggleDownload(state.repoWithTag)}
+ stopPropagationOnClick
+ tooltip={state.isPaused ? 'Resume downloading' : 'Pause downloading'}
+ tooltipAsTitle
+ />
+
+ <ActionIcon
+ class="text-muted-foreground hover:text-destructive"
+ icon={X}
+ iconSize="h-4 w-4"
+ onclick={() => state.onRequestCancel?.(state.repoWithTag)}
+ stopPropagationOnClick
+ tooltip="Cancel downloading"
+ tooltipAsTitle
+ />
+ </span>
+ </span>
+ {/if}
+ </div>
+
+ <!-- only with sub-lines, so a plain row is the row's own height and the
+ progress bar overlays its bottom edge -->
+ {#if onDiskDrafts.length}
+ <div class="flex flex-col px-2.5 pb-1.5">
+ {#each onDiskDrafts as badge (badge.repo + ':' + badge.kind + ':' + (badge.quant ?? ''))}
+ <ModelDraftSubline
+ connectorClass="ml-8"
+ kind={badge.kind}
+ quant={badge.quant}
+ state={ModelRowDownloadState.DOWNLOADED}
+ />
+ {/each}
+ </div>
+ {/if}
+
+ {#if download?.progress && download.progress.totalBytes > 0}
+ <ModelDownloadProgressBar
+ downloadedBytes={download.progress.downloadedBytes}
+ overlay
+ totalBytes={download.progress.totalBytes}
+ />
+ {/if}
+</li>
diff --git a/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverList/ModelsDiscoverListItemSkeleton.svelte b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverList/ModelsDiscoverListItemSkeleton.svelte
new file mode 100644
index 000000000..851abc523
--- /dev/null
+++ b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverList/ModelsDiscoverListItemSkeleton.svelte
@@ -0,0 +1,39 @@
+<script lang="ts">
+ import { Skeleton } from '$lib/components/ui/skeleton';
+
+ interface Props {
+ /** Row index; varies skeleton widths so the list does not look mechanical. */
+ index?: number;
+ }
+
+ let { index = 0 }: Props = $props();
+
+ // Deterministic width variations keyed by row position.
+ const NAME_WIDTHS = ['w-40', 'w-52', 'w-44', 'w-56'];
+ const BADGE_WIDTHS = [
+ ['w-12', 'w-14', 'w-10'],
+ ['w-16', 'w-12', 'w-12'],
+ ['w-10', 'w-16', 'w-10'],
+ ['w-14', 'w-10', 'w-14']
+ ];
+
+ let nameWidth = $derived(NAME_WIDTHS[index % NAME_WIDTHS.length]);
+ let badgeWidths = $derived(BADGE_WIDTHS[index % BADGE_WIDTHS.length]);
+</script>
+
+<!-- Static skeleton of ModelsDiscoverListItem: avatar, name and badge rows. -->
+<li>
+ <div class="flex w-full items-start gap-2.5 rounded-lg p-2.5 text-left">
+ <Skeleton class="h-9 w-9 shrink-0 rounded-md" />
+
+ <div class="min-w-0 flex-1 space-y-1.5">
+ <Skeleton class="{nameWidth} h-4 max-w-full" />
+
+ <div class="flex items-center gap-1">
+ {#each badgeWidths as width, i (i)}
+ <Skeleton class="{width} h-3.5 rounded" />
+ {/each}
+ </div>
+ </div>
+ </div>
+</li>
diff --git a/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverList/ModelsDiscoverListSearch.svelte b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverList/ModelsDiscoverListSearch.svelte
new file mode 100644
index 000000000..458c31648
--- /dev/null
+++ b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverList/ModelsDiscoverListSearch.svelte
@@ -0,0 +1,30 @@
+<script lang="ts">
+ import { SearchInput } from '$lib/components/app';
+
+ interface Props {
+ value?: string;
+ /** Search callback; the parent owns debouncing. */
+ onSearch?: (query: string) => void;
+ placeholder?: string;
+ }
+
+ let {
+ onSearch,
+ placeholder = 'Search models on Hugging Face',
+ value = $bindable('')
+ }: Props = $props();
+
+ function handleInput(next: string) {
+ value = next;
+
+ onSearch?.(value);
+ }
+</script>
+
+<!-- Inset from the panel edges so the focus ring has room inside the scroll
+ container, and opaque so rows scroll behind the bar instead of through it.
+ Padding (not margin) for the top room: a sticky margin keeps a gap when
+ stuck, where rows would peek through. -->
+<div class="sticky top-0 z-99 pt-1.5">
+ <SearchInput bind:value onInput={(v) => handleInput(v)} {placeholder} size="sm" />
+</div>
diff --git a/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverList/index.ts b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverList/index.ts
new file mode 100644
index 000000000..c3025c0b2
--- /dev/null
+++ b/tools/ui/src/lib/components/app/models/discover/ModelsDiscoverList/index.ts
@@ -0,0 +1,16 @@
+/**
+ *
+ * MODELS DISCOVER - LIST
+ *
+ * The discover sidebar column: a debounced search field above a navigable list of
+ * HuggingFace GGUF models, with skeleton rows while loading.
+ *
+ */
+
+export { default as ModelsDiscoverList } from './ModelsDiscoverList.svelte';
+export { default as ModelsDiscoverListSearch } from './ModelsDiscoverListSearch.svelte';
+
+/** One model entry; also the row of a tracked download, with its own controls. */
+export { default as ModelsDiscoverListItem } from './ModelsDiscoverListItem.svelte';
+
+export { default as ModelsDiscoverListItemSkeleton } from './ModelsDiscoverListItemSkeleton.svelte';
diff --git a/tools/ui/src/lib/components/app/models/discover/index.ts b/tools/ui/src/lib/components/app/models/discover/index.ts
new file mode 100644
index 000000000..bb353c72d
--- /dev/null
+++ b/tools/ui/src/lib/components/app/models/discover/index.ts
@@ -0,0 +1,16 @@
+/**
+ *
+ * MODELS DISCOVER
+ *
+ * Components for the Models Discover view: a sidebar search + list of HuggingFace GGUF
+ * models and a detail view for the selected model, used as the body of the discovery
+ * dialog. The list and detail trees live in their own subfolders; this barrel re-exports
+ * them alongside the shared leaves.
+ *
+ */
+
+/** The whole view: sidebar list on the left, detail pane on the right. */
+export { default as ModelsDiscover } from './ModelsDiscover.svelte';
+
+export * from './ModelsDiscoverList';
+export * from './ModelsDiscoverDetails';
diff --git a/tools/ui/src/lib/components/app/models/index.ts b/tools/ui/src/lib/components/app/models/index.ts
index dbfb5c60e..83f076fd1 100644
--- a/tools/ui/src/lib/components/app/models/index.ts
+++ b/tools/ui/src/lib/components/app/models/index.ts
@@ -59,4 +59,7 @@ export { default as ModelCapabilityIcons } from './ModelCapabilityIcons.svelte';
/** Draft sidecar badges, rendered as `+ [KIND] [QUANT]` per sidecar. */
export { default as ModelDraftSidecars } from './ModelDraftSidecars.svelte';
+
+/** Compact draft download line, embedded in the item it belongs to. */
+export { default as ModelDraftSubline } from './ModelDraftSubline.svelte';
export * from './ModelsSelector';
diff --git a/tools/ui/src/lib/components/app/settings/SettingsChat/SettingsChat.svelte b/tools/ui/src/lib/components/app/settings/SettingsChat/SettingsChat.svelte
index 879390075..aebf37251 100644
--- a/tools/ui/src/lib/components/app/settings/SettingsChat/SettingsChat.svelte
+++ b/tools/ui/src/lib/components/app/settings/SettingsChat/SettingsChat.svelte
@@ -13,6 +13,7 @@
NUMERIC_FIELDS,
POSITIVE_INTEGER_FIELDS,
SETTINGS_CHAT_SECTIONS,
+ SETTINGS_KEYS,
SETTINGS_SECTION_SLUGS
} from '$lib/constants';
import { ColorMode } from '$lib/enums/ui.enums';
@@ -70,6 +71,12 @@
function handleConfigChange(key: string, value: string | boolean) {
localConfig[key] = value;
+
+ // discover needs the hub metadata reads: checking it turns the hub on,
+ // unchecking it turns the hub back off, so the checkbox never lies
+ if (key === SETTINGS_KEYS.ENABLE_DISCOVER_MODELS) {
+ localConfig[SETTINGS_KEYS.USE_HUGGING_FACE_HUB] = value === true;
+ }
}
function handleReset() {
diff --git a/tools/ui/src/lib/components/app/settings/SettingsChat/SettingsChatFields.svelte b/tools/ui/src/lib/components/app/settings/SettingsChat/SettingsChatFields.svelte
index e8795d7c5..5dfba6bb3 100644
--- a/tools/ui/src/lib/components/app/settings/SettingsChat/SettingsChatFields.svelte
+++ b/tools/ui/src/lib/components/app/settings/SettingsChat/SettingsChatFields.svelte
@@ -22,6 +22,13 @@
let { fields, localConfig, onConfigChange, onThemeChange }: Props = $props();
+ // discover needs the hub metadata reads, so its checkbox is read-only while discover is on
+ let hubLocked = $derived(Boolean(localConfig[SETTINGS_KEYS.ENABLE_DISCOVER_MODELS]));
+
+ function isLockedField(key: string): boolean {
+ return hubLocked && key === SETTINGS_KEYS.USE_HUGGING_FACE_HUB;
+ }
+
let currentModelParams = $derived.by(() => {
void modelsStore.props.cacheVersion;
@@ -301,13 +308,18 @@
<Checkbox
checked={Boolean(localConfig[field.key])}
class="mt-1"
+ disabled={isLockedField(field.key)}
id={field.key}
onCheckedChange={(checked) => onConfigChange(field.key, checked)}
/>
<div class="space-y-1">
<label
- class="flex cursor-pointer items-center gap-1.5 pt-1 pb-0.5 text-sm leading-none font-medium"
+ class="flex items-center gap-1.5 pt-1 pb-0.5 text-sm leading-none font-medium {isLockedField(
+ field.key
+ )
+ ? 'cursor-default opacity-80'
+ : 'cursor-pointer'}"
for={field.key}
>
{field.label}
diff --git a/tools/ui/src/lib/components/ui/select/select-content.svelte b/tools/ui/src/lib/components/ui/select/select-content.svelte
index f8792f8a9..a8263819a 100644
--- a/tools/ui/src/lib/components/ui/select/select-content.svelte
+++ b/tools/ui/src/lib/components/ui/select/select-content.svelte
@@ -88,7 +88,7 @@
<SelectPrimitive.Content
bind:ref
class={cn(
- 'relative z-[var(--layer-popover,1000000)] max-h-(--bits-select-content-available-height) min-w-[8rem] origin-(--bits-select-content-transform-origin) overflow-x-hidden overflow-y-auto rounded-md border bg-popover text-popover-foreground shadow-md data-[side=bottom]:translate-y-1 data-[side=bottom]:slide-in-from-top-2 data-[side=left]:-translate-x-1 data-[side=left]:slide-in-from-right-2 data-[side=right]:translate-x-1 data-[side=right]:slide-in-from-left-2 data-[side=top]:-translate-y-1 data-[side=top]:slide-in-from-bottom-2 data-[state=closed]:animate-out data-[state=closed]:fade-out-0 data-[state=closed]:fill-mode-forwards data-[state=closed]:zoom-out-95 data-[state=open]:animate-in data-[state=open]:fade-in-0 data-[state=open]:zoom-in-95',
+ 'relative z-[var(--layer-popover,1000000)] max-h-(--bits-select-content-available-height) min-w-[8rem] origin-(--bits-select-content-transform-origin) overflow-x-hidden overflow-y-auto rounded-md border border-border bg-popover dark:border-border/20 text-popover-foreground shadow-md data-[side=bottom]:translate-y-1 data-[side=bottom]:slide-in-from-top-2 data-[side=left]:-translate-x-1 data-[side=left]:slide-in-from-right-2 data-[side=right]:translate-x-1 data-[side=right]:slide-in-from-left-2 data-[side=top]:-translate-y-1 data-[side=top]:slide-in-from-bottom-2 data-[state=closed]:animate-out data-[state=closed]:fade-out-0 data-[state=closed]:fill-mode-forwards data-[state=closed]:zoom-out-95 data-[state=open]:animate-in data-[state=open]:fade-in-0 data-[state=open]:zoom-in-95',
className
)}
data-slot="select-content"
@@ -99,7 +99,7 @@
<SelectPrimitive.Viewport
class={cn(
- 'h-(--bits-select-anchor-height) w-full min-w-(--bits-select-anchor-width) scroll-my-1 p-1'
+ 'h-(--bits-select-anchor-height) w-full min-w-(--bits-select-anchor-width) scroll-my-1 p-1.5'
)}
>
{@render children?.()}
diff --git a/tools/ui/src/lib/components/ui/select/select-trigger.svelte b/tools/ui/src/lib/components/ui/select/select-trigger.svelte
index 9f642819e..de3abcdf2 100644
--- a/tools/ui/src/lib/components/ui/select/select-trigger.svelte
+++ b/tools/ui/src/lib/components/ui/select/select-trigger.svelte
@@ -11,20 +11,26 @@
variant = 'default',
...restProps
}: WithoutChild<SelectPrimitive.TriggerProps> & {
- size?: 'sm' | 'default';
+ size?: 'xs' | 'sm' | 'default';
variant?: 'default' | 'plain';
} = $props();
+ // Super small trigger: fits its selected value, for dense inline use.
+ const xsClasses =
+ "group flex h-6 w-fit items-center justify-between gap-1 rounded-md border border-border/30 bg-muted/60 px-2 py-0 text-xs whitespace-nowrap outline-none select-none transition-colors hover:bg-muted/80 focus-visible:border-ring focus-visible:ring-[3px] focus-visible:ring-ring/50 disabled:cursor-not-allowed disabled:opacity-50 data-[placeholder]:text-muted-foreground dark:border-border/20 dark:bg-muted/75 dark:hover:bg-muted [&_svg]:pointer-events-none [&_svg]:shrink-0 [&_svg:not([class*='text-'])]:text-muted-foreground";
+
const baseClasses = $derived(
variant === 'plain'
? "group inline-flex w-full items-center justify-end gap-2 whitespace-nowrap px-0 py-0 text-sm font-medium text-muted-foreground transition-colors focus-visible:outline-none focus-visible:ring-0 focus-visible:ring-offset-0 disabled:cursor-not-allowed disabled:opacity-50 data-[placeholder]:text-muted-foreground data-[size=default]:h-9 data-[size=sm]:h-8 [&_svg]:pointer-events-none [&_svg]:shrink-0 [&_svg:not([class*='size-'])]:size-3 [&_svg:not([class*='text-'])]:text-muted-foreground"
- : "flex w-fit items-center justify-between gap-2 rounded-md border border-input bg-transparent px-3 py-2 text-sm whitespace-nowrap shadow-xs transition-[color,box-shadow] outline-none select-none focus-visible:border-ring focus-visible:ring-[3px] focus-visible:ring-ring/50 disabled:cursor-not-allowed disabled:opacity-50 aria-invalid:border-destructive aria-invalid:ring-destructive/20 data-[placeholder]:text-muted-foreground data-[size=default]:h-9 data-[size=sm]:h-8 *:data-[slot=select-value]:line-clamp-1 *:data-[slot=select-value]:flex *:data-[slot=select-value]:items-center *:data-[slot=select-value]:gap-2 dark:bg-input/30 dark:hover:bg-input/50 dark:aria-invalid:ring-destructive/40 [&_svg]:pointer-events-none [&_svg]:shrink-0 [&_svg:not([class*='size-'])]:size-4 [&_svg:not([class*='text-'])]:text-muted-foreground"
+ : size === 'xs'
+ ? xsClasses
+ : "flex w-fit items-center justify-between gap-2 rounded-md border border-border/30 bg-muted/60 px-3 py-2 text-sm whitespace-nowrap shadow-sm transition-colors outline-none select-none hover:bg-muted/80 focus-visible:border-ring focus-visible:ring-[3px] focus-visible:ring-ring/50 disabled:cursor-not-allowed disabled:opacity-50 aria-invalid:border-destructive aria-invalid:ring-destructive/20 data-[placeholder]:text-muted-foreground data-[size=default]:h-9 data-[size=sm]:h-8 *:data-[slot=select-value]:line-clamp-1 *:data-[slot=select-value]:flex *:data-[slot=select-value]:items-center *:data-[slot=select-value]:gap-2 dark:border-border/20 dark:bg-muted/75 dark:hover:bg-muted dark:aria-invalid:ring-destructive/40 [&_svg]:pointer-events-none [&_svg]:shrink-0 [&_svg:not([class*='size-'])]:size-3.5 [&_svg:not([class*='text-'])]:text-muted-foreground"
);
const chevronClasses = $derived(
- variant === 'plain'
+ variant === 'plain' || size === 'xs'
? 'size-3 opacity-60 transition-transform group-data-[state=open]:-rotate-180'
- : 'size-4 opacity-50'
+ : 'size-3.5 opacity-60 transition-transform group-data-[state=open]:-rotate-180'
);
</script>
diff --git a/tools/ui/src/lib/constants/huggingface.constants.ts b/tools/ui/src/lib/constants/huggingface.constants.ts
index 39f6fbe5c..47b6a89be 100644
--- a/tools/ui/src/lib/constants/huggingface.constants.ts
+++ b/tools/ui/src/lib/constants/huggingface.constants.ts
@@ -99,6 +99,12 @@ export const HF_GATED_TAG = 'gated';
export const HF_GGUF_TAG = 'gguf';
export const HF_SAFETENSORS_TAG = 'safetensors';
+/** Hub tag marking decision models, which answer with probabilities, not text. */
+export const HF_DECISION_MODEL_TAG = 'decision-model';
+
+/** `pipeline_tag` that also marks a decision model, with or without the hub tag. */
+export const HF_DECISION_PIPELINE_TAG = 'text-classification';
+
// Pipeline tasks (logic use only - matching `pipeline_tag` values against tags)
/**
diff --git a/tools/ui/src/lib/constants/icons.constants.ts b/tools/ui/src/lib/constants/icons.constants.ts
index 54a143330..8ac041c64 100644
--- a/tools/ui/src/lib/constants/icons.constants.ts
+++ b/tools/ui/src/lib/constants/icons.constants.ts
@@ -5,6 +5,7 @@
import {
CircleAlert as LoadFailedIcon,
+ Compass as DiscoverIcon,
Eject as UnloadIcon,
Eye as VisionIcon,
File as FileIcon,
@@ -45,6 +46,9 @@ export const DEFAULT_FILE_ICON = FileIcon;
/** The mark for a model, wherever one is listed. */
export const MODEL_ICON = ModelIcon;
+/** The mark for the Discover Models surface and its entry points. */
+export const DISCOVER_ICON = DiscoverIcon;
+
export const MODALITY_ICONS = {
[ModelModality.AUDIO]: AudioIcon,
[ModelModality.VIDEO]: VideoIcon,
diff --git a/tools/ui/src/lib/constants/index.ts b/tools/ui/src/lib/constants/index.ts
index 6f795634b..afbad3479 100644
--- a/tools/ui/src/lib/constants/index.ts
+++ b/tools/ui/src/lib/constants/index.ts
@@ -48,6 +48,7 @@ export * from './model-id.constants';
export * from './model-loading.constants';
export * from './models-manager.constants';
export * from './models-discover.constants';
+export * from './models-discover-download.constants';
export * from './model-compatibility.constants';
export * from './huggingface.constants';
export * from './precision.constants';
diff --git a/tools/ui/src/lib/constants/models-discover-download.constants.ts b/tools/ui/src/lib/constants/models-discover-download.constants.ts
new file mode 100644
index 000000000..721d46db3
--- /dev/null
+++ b/tools/ui/src/lib/constants/models-discover-download.constants.ts
@@ -0,0 +1,41 @@
+/**
+ * Models Discover - download options constants: the serve `--spec-type` mapping,
+ * the bit-depth buckets, and the standalone command builder.
+ */
+
+import { ModelDraftSidecar } from '$lib/enums';
+
+/**
+ * `--spec-type` value for each draft sidecar.
+ */
+export const SPEC_TYPE: Record<ModelDraftSidecar, string> = {
+ [ModelDraftSidecar.DFLASH]: 'draft-dflash',
+ [ModelDraftSidecar.DSPARK]: 'draft-dspark',
+ [ModelDraftSidecar.EAGLE3]: 'draft-eagle3',
+ [ModelDraftSidecar.MTP]: 'draft-mtp'
+};
+
+/** Bit-depth bucket for files that carry no quant token; rendered as "Other". */
+export const OTHER_BIT_DEPTH = 99;
+
+export const OTHER_BIT_DEPTH_LABEL = 'Other';
+
+/** Suffix appended after a bit-depth number to label a row, e.g. `4-bit`. */
+export const BIT_DEPTH_LABEL_SUFFIX = '-bit';
+
+/**
+ * Fixed tokens of the standalone `llama serve` command preview, so the command
+ * builder and its rendered spans share one source of the CLI spelling.
+ */
+export const SERVE_COMMAND = {
+ BIN: 'llama',
+ MODEL_FLAG: '-hf',
+ SPEC_TYPE_FLAG: '--spec-type',
+ SUBCOMMAND: 'serve'
+} as const;
+
+/** Bit depth preferred for the command's default base quant. */
+export const DEFAULT_BASE_BIT_DEPTH = 4;
+
+/** Label shown for a draft-sidecar chip whose quant is unknown. */
+export const DRAFT_FILE_LABEL = 'draft';
diff --git a/tools/ui/src/lib/constants/models-discover.constants.ts b/tools/ui/src/lib/constants/models-discover.constants.ts
index f2a6a9463..b0a0abc56 100644
--- a/tools/ui/src/lib/constants/models-discover.constants.ts
+++ b/tools/ui/src/lib/constants/models-discover.constants.ts
@@ -6,3 +6,9 @@
/** llama.app model catalog used as the default model list. Online-only source; the discover feature requires an internet connection anyway. */
export const MODELS_DISCOVER_CATALOG_URL = 'https://llama.app/v1/catalog.json';
+
+/**
+ * Catalog repos fetched in parallel per batch; small on purpose so the HF API
+ * is not hit with the whole catalog at once.
+ */
+export const MODELS_DISCOVER_CATALOG_BATCH = 4;
diff --git a/tools/ui/src/lib/constants/settings-keys.constants.ts b/tools/ui/src/lib/constants/settings-keys.constants.ts
index 0cbda109d..d4ae8318c 100644
--- a/tools/ui/src/lib/constants/settings-keys.constants.ts
+++ b/tools/ui/src/lib/constants/settings-keys.constants.ts
@@ -26,6 +26,7 @@ export const SETTINGS_KEYS = {
DYNATEMP_EXPONENT: 'dynatemp_exponent',
DYNATEMP_RANGE: 'dynatemp_range',
ENABLE_CONTINUE_GENERATION: 'enableContinueGeneration',
+ ENABLE_DISCOVER_MODELS: 'enableDiscoverModels',
EXCLUDE_REASONING_FROM_CONTEXT: 'excludeReasoningFromContext',
FREQUENCY_PENALTY: 'frequency_penalty',
FULL_HEIGHT_CODE_BLOCKS: 'fullHeightCodeBlocks',
diff --git a/tools/ui/src/lib/constants/settings.constants.ts b/tools/ui/src/lib/constants/settings.constants.ts
index 9d01f3ad0..9d00ac9ef 100644
--- a/tools/ui/src/lib/constants/settings.constants.ts
+++ b/tools/ui/src/lib/constants/settings.constants.ts
@@ -127,12 +127,17 @@ export const SETTINGS_REGISTRY: SettingsSectionEntry[] = [
type: SettingsFieldType.CHECKBOX
},
{
- // Deliberately off for now: the natural place to turn it on is the first
- // run experience, once onboarding exists to ask the user about it.
defaultValue: false,
- help: 'Fetch model metadata (avatars, context length, chat template) from the Hugging Face Hub. When off, the UI only shows what the server reports for /v1/models and hides the org avatars.',
+ help: 'Show the Discover Models sidebar action to browse and download HuggingFace GGUF models. Turns the Hugging Face Hub API on while it is enabled. Only available in router mode.',
+ key: SETTINGS_KEYS.ENABLE_DISCOVER_MODELS,
+ label: 'Enable Discover Models',
+ type: SettingsFieldType.CHECKBOX
+ },
+ {
+ defaultValue: false,
+ help: 'Fetch model display data (avatars, context length, chat template) and everything in the Discover Models view from the Hugging Face Hub API. Locked on while Discover Models is enabled; can be turned on alone for just the display data.',
key: SETTINGS_KEYS.USE_HUGGING_FACE_HUB,
- label: 'Use Hugging Face Hub API for models metadata',
+ label: 'Use the Hugging Face Hub API',
type: SettingsFieldType.CHECKBOX
},
{
diff --git a/tools/ui/src/lib/constants/storage.constants.ts b/tools/ui/src/lib/constants/storage.constants.ts
index c81b30613..637a1e0e8 100644
--- a/tools/ui/src/lib/constants/storage.constants.ts
+++ b/tools/ui/src/lib/constants/storage.constants.ts
@@ -16,8 +16,6 @@ export const DB_APP_NAME_DEPRECATED = 'LlamacppWebui';
export const ALWAYS_ALLOWED_TOOLS_LOCALSTORAGE_KEY = `${STORAGE_APP_NAME}.alwaysAllowedTools`;
-/** Paused model download ids (`<repo>:<tag>`), restored on the next page load. */
-export const PAUSED_MODEL_DOWNLOADS_LOCALSTORAGE_KEY = `${STORAGE_APP_NAME}.pausedModelDownloads`;
export const CONFIG_LOCALSTORAGE_KEY = `${STORAGE_APP_NAME}.config`;
export const DISABLED_TOOLS_LOCALSTORAGE_KEY = `${STORAGE_APP_NAME}.disabledTools`;
diff --git a/tools/ui/src/lib/constants/ui.constants.ts b/tools/ui/src/lib/constants/ui.constants.ts
index 4b43cceee..946793978 100644
--- a/tools/ui/src/lib/constants/ui.constants.ts
+++ b/tools/ui/src/lib/constants/ui.constants.ts
@@ -4,6 +4,12 @@ import McpLogo from '$lib/components/app/mcp/McpLogo.svelte';
import { SidebarAction, ToolSource } from '$lib/enums';
import type { DesktopIconStripItem } from '$lib/types';
+/** Delay before a copied affordance resets from its check to the copy icon. */
+export const COPY_FEEDBACK_RESET_MS = 1500;
+
+/** How long a models dialog view fades out before the next one fades in. */
+export const MODELS_DIALOG_VIEW_FADE_MS = 150;
+
export const FORK_TREE_DEPTH_PADDING = 8;
export const SYSTEM_MESSAGE_PLACEHOLDER = 'System message';
diff --git a/tools/ui/src/lib/enums/index.ts b/tools/ui/src/lib/enums/index.ts
index 6843b0a19..9ab0be32e 100644
--- a/tools/ui/src/lib/enums/index.ts
+++ b/tools/ui/src/lib/enums/index.ts
@@ -69,7 +69,13 @@ export {
JsonSchemaType
} from './mcp.enums';
-export { ModelAuxSidecar, ModelCapability, ModelDraftSidecar, ModelModality } from './model.enums';
+export {
+ ModelAuxSidecar,
+ ModelCapability,
+ ModelDraftSidecar,
+ ModelModality,
+ ModelSelectableFileKind
+} from './model.enums';
export {
ModelDownloadConfirmAction,
@@ -89,6 +95,7 @@ export {
ColorMode,
HtmlInputType,
McpPromptVariant,
+ ModelsDialogView,
ScrollCarouselVariant,
SidebarAction,
TooltipSide,
diff --git a/tools/ui/src/lib/enums/model.enums.ts b/tools/ui/src/lib/enums/model.enums.ts
index e707fe169..2f25c26f4 100644
--- a/tools/ui/src/lib/enums/model.enums.ts
+++ b/tools/ui/src/lib/enums/model.enums.ts
@@ -37,9 +37,18 @@ export enum ModelAuxSidecar {
}
/**
- * Why an in-flight download is being stopped, so the terminal `download_failed`
- * feed event can be attributed: a user pause (resumable) or a user cancel
- * (discard). Distinguishes these from a genuine download failure.
+ * Role of a selectable GGUF in the download options: the main weights, a
+ * speculative-decoding draft sidecar, or an auxiliary sidecar (mmproj).
+ */
+export enum ModelSelectableFileKind {
+ AUX = 'aux',
+ DRAFT = 'draft',
+ MAIN = 'main'
+}
+
+/**
+ * Why an in-flight download is being stopped, so the terminal `download_failed` feed
+ * event can be attributed: a user pause (resumable) or a user cancel (discard).
*/
export enum ModelDownloadStopRequest {
CANCEL = 'cancel',
@@ -73,6 +82,7 @@ export enum ModelsTableGroupKind {
/** Download state a manager row reports in place of its load state. */
export enum ModelRowDownloadState {
+ DOWNLOADED = 'downloaded',
DOWNLOADING = 'downloading',
PAUSED = 'paused'
}
diff --git a/tools/ui/src/lib/enums/server.enums.ts b/tools/ui/src/lib/enums/server.enums.ts
index b558b5123..6f9058942 100644
--- a/tools/ui/src/lib/enums/server.enums.ts
+++ b/tools/ui/src/lib/enums/server.enums.ts
@@ -18,6 +18,8 @@ export enum ServerModelStatus {
DOWNLOADED = 'downloaded',
DOWNLOADING = 'downloading',
FAILED = 'failed',
+ /** A download that never finished: listed by the server without an instance, resumable. */
+ INCOMPLETE = 'incomplete',
LOADED = 'loaded',
LOADING = 'loading',
SLEEPING = 'sleeping',
diff --git a/tools/ui/src/lib/enums/ui.enums.ts b/tools/ui/src/lib/enums/ui.enums.ts
index 20cfe47ff..599fc2128 100644
--- a/tools/ui/src/lib/enums/ui.enums.ts
+++ b/tools/ui/src/lib/enums/ui.enums.ts
@@ -32,6 +32,12 @@ export enum SidebarAction {
/**
* MCP prompt display variant
*/
+/** View the models dialog renders. */
+export enum ModelsDialogView {
+ DISCOVER = 'discover',
+ MANAGE = 'manage'
+}
+
export enum McpPromptVariant {
ATTACHMENT = 'attachment',
MESSAGE = 'message'
diff --git a/tools/ui/src/lib/services/huggingface.service.ts b/tools/ui/src/lib/services/huggingface.service.ts
index 2d153f718..4cea6eb62 100644
--- a/tools/ui/src/lib/services/huggingface.service.ts
+++ b/tools/ui/src/lib/services/huggingface.service.ts
@@ -15,6 +15,8 @@ import {
HF_BASE_URL,
HF_CACHE_DIR_SEPARATOR,
HF_CACHE_PATH_REGEX,
+ HF_DECISION_MODEL_TAG,
+ HF_DECISION_PIPELINE_TAG,
HF_DEFAULT_LIMIT,
HF_FIRST_SHARD,
HF_FRONTMATTER_REGEX,
@@ -144,6 +146,12 @@ export class HuggingFaceService {
Q8_0: 8
};
+ // File trees already fetched, keyed by repo id: the discover view asks for the same
+ // repo from several places.
+ private static treeCache = new Map<string, HfModelSibling[]>();
+
+ private static treePending = new Map<string, Promise<HfModelSibling[]>>();
+
/** Details already fetched, for a caller that cannot await (a table sorting its rows). */
static cachedDetails(modelId: string): HfModelDetailInfo | null | undefined {
// llama.cpp model ids carry the quant tag after a colon
@@ -152,6 +160,8 @@ export class HuggingFaceService {
return HuggingFaceService.detailsCache.get(hfRepoId);
}
+ // GGUF Model Browsing
+
/**
* Collapse split GGUF shard sets (`-00001-of-00015.gguf`, ...) to their first
* shard, summing every shard's size so the kept entry reflects the whole
@@ -190,8 +200,6 @@ export class HuggingFaceService {
return result;
}
- // GGUF Model Browsing
-
/**
* Extract the GGUF quantization token (e.g. `Q4_K_M`) and any sidecar type
* (`mtp`, `dflash`, `mmproj`, ...) from a `.gguf` filename.
@@ -347,6 +355,8 @@ export class HuggingFaceService {
return `${Math.floor(diffDays / DAYS_PER_YEAR)} ${YEARS_AGO_LABEL}`;
}
+ // Model Details & Files
+
/**
* Format a min-max size range with one shared unit, e.g. `19.0-28.6 GB`.
*/
@@ -372,8 +382,6 @@ export class HuggingFaceService {
return `${fmt(min)}-${fmt(max)} ${unit}`;
}
- // Model Details & Files
-
/**
* Avatar URL for an author (org or user). 404s when the author does not
* exist, so callers should provide a fallback.
@@ -476,7 +484,9 @@ export class HuggingFaceService {
if (!response.ok) throw new Error(`Failed to fetch catalog: ${response.status}`);
- return (await response.json()) as HfCatalogEntry[];
+ const entries = (await response.json()) as HfCatalogEntry[];
+
+ return entries.filter((entry) => !entry.decision);
}
static getDetails(modelId: string): Promise<HfModelDetailInfo | null> {
@@ -538,12 +548,12 @@ export class HuggingFaceService {
return `${HF_BASE_URL}${PATH_SEPARATOR}${modelId}`;
}
+ // Utility Methods
+
static async getMostLiked(limit: number = HF_DEFAULT_LIMIT): Promise<HfModelInfo[]> {
return this.search({ limit, sort: HfModelSort.LIKES });
}
- // Utility Methods
-
static async getNew(limit: number = HF_DEFAULT_LIMIT): Promise<HfModelInfo[]> {
return this.search({ limit, sort: HfModelSort.CREATED_AT });
}
@@ -576,26 +586,30 @@ export class HuggingFaceService {
}
}
- /**
- * Get repository file tree to list available GGUF variants. Recursive so
- * repos that keep quants in per-quant subdirectories (e.g. `UD-Q4_K_XL/`)
- * are included; follows cursor pagination for repos over one page.
- */
- static async getTree(modelId: string): Promise<HfModelSibling[]> {
- if (!HuggingFaceService.isEnabled()) return [];
+ static getTree(modelId: string): Promise<HfModelSibling[]> {
+ if (!HuggingFaceService.isEnabled()) return Promise.resolve([]);
- const files: HfModelSibling[] = [];
- const firstUrl =
- `${HF_API_MODELS_URL}${PATH_SEPARATOR}${modelId}${PATH_SEPARATOR}${HF_TREE_PATH}` +
- `${PATH_SEPARATOR}${HF_MAIN_BRANCH}?${HF_RECURSIVE_TREE_PARAM}`;
+ const cached = HuggingFaceService.treeCache.get(modelId);
- let url: string | null = firstUrl;
+ if (cached) return Promise.resolve(cached);
+
+ const pending = HuggingFaceService.treePending.get(modelId);
+
+ if (pending) return pending;
+
+ const promise = (async () => {
+ const files: HfModelSibling[] = [];
+ const firstUrl =
+ `${HF_API_MODELS_URL}${PATH_SEPARATOR}${modelId}${PATH_SEPARATOR}${HF_TREE_PATH}` +
+ `${PATH_SEPARATOR}${HF_MAIN_BRANCH}?${HF_RECURSIVE_TREE_PARAM}`;
+
+ let url: string | null = firstUrl;
- try {
for (let page = 0; url && page < HF_TREE_MAX_PAGES; page++) {
const response: Response = await fetch(url);
- if (!response.ok) return files;
+ if (!response.ok)
+ throw new Error(`Failed to fetch tree for ${modelId}: ${response.status}`);
const data = (await response.json()) as HfModelSibling[];
@@ -603,11 +617,24 @@ export class HuggingFaceService {
url = HuggingFaceService.parseNextPageUrl(response.headers.get(HF_LINK_HEADER));
}
- } catch {
- // Return whatever was fetched before the failure.
- }
- return files;
+ HuggingFaceService.treeCache.set(modelId, files);
+
+ return files;
+ })()
+ .catch((error: unknown) => {
+ // not cached: a rate limited or failed fetch should retry on the
+ // next mount; an empty tree makes the store fall back to the
+ // catalog's advertised sizes
+ console.error(`Error fetching tree for ${modelId}:`, error);
+
+ return [] as HfModelSibling[];
+ })
+ .finally(() => HuggingFaceService.treePending.delete(modelId));
+
+ HuggingFaceService.treePending.set(modelId, promise);
+
+ return promise;
}
static async getTrending(limit: number = HF_DEFAULT_LIMIT): Promise<HfModelInfo[]> {
@@ -615,9 +642,21 @@ export class HuggingFaceService {
}
/**
- * True when the UI may read model metadata from the Hugging Face Hub. Off by
- * default: with it off, callers only see what the server's /v1/models reports
- * and the org avatars stay hidden.
+ * True for decision models: they answer with calibrated probabilities instead
+ * of generated text, so llama-ui cannot use them and they stay out of every
+ * list the Hub data layer returns.
+ */
+ static isDecisionModel(model: Pick<HfModelInfo, 'pipeline_tag' | 'tags'>): boolean {
+ return (
+ model.tags.includes(HF_DECISION_MODEL_TAG) || model.pipeline_tag === HF_DECISION_PIPELINE_TAG
+ );
+ }
+
+ /**
+ * True when the UI may read from the Hugging Face Hub API: model display data
+ * (avatars, context length, chat template) and the Discover Models view. With
+ * it off, callers only see what the server's /v1/models reports and the org
+ * avatars stay hidden.
*/
static isEnabled(): boolean {
return settingsStore.config[SETTINGS_KEYS.USE_HUGGING_FACE_HUB] ?? false;
@@ -719,8 +758,9 @@ export class HuggingFaceService {
filter: HF_GGUF_FILTER,
limit: Math.min(limit, HF_MAX_LIMIT)
});
+ const models = await this.fetchWithRetry(url);
- return this.fetchWithRetry(url);
+ return models.filter((model) => !HuggingFaceService.isDecisionModel(model));
}
static async searchByQuery(
diff --git a/tools/ui/src/lib/services/models.service.ts b/tools/ui/src/lib/services/models.service.ts
index 96902a0b9..e1882116d 100644
--- a/tools/ui/src/lib/services/models.service.ts
+++ b/tools/ui/src/lib/services/models.service.ts
@@ -9,15 +9,16 @@
import { base } from '$app/paths';
import { API_MODELS, MODEL_ID, type ModelSidecar } from '$lib/constants';
import { ServerModelStatus } from '$lib/enums';
-import type { ParsedModelId } from '$lib/types/models';
+import type { ModelSidecarBadge, ParsedModelId } from '$lib/types/models';
import {
apiDelete,
apiFetch,
apiPost,
extractSseDataPayload,
+ isAuxSidecar,
normalizeModelName,
+ repoOf,
sidecarFromFileToken,
- sidecarFromTag,
splitSseRecords
} from '$lib/utils';
import { getAuthHeaders } from '$lib/utils/api-headers';
@@ -26,8 +27,9 @@ export class ModelsService {
private static readonly SSE_RECONNECT_MS = 1000;
/**
- * Build the `<repo>:<tag>` string POST /models expects, so callers don't need
- * to know the tag conventions.
+ * Build the `<repo>:<tag>` id of a download option: `<quant>` for the model
+ * weights, `<quant>-<sidecar>` for a sidecar of that quant. The UI keys its
+ * options with it; the server only ever sees the model and its sidecar kinds.
*/
static buildDownloadTag(
repoId: string,
@@ -44,26 +46,72 @@ export class ModelsService {
}
/**
- * Cancel an in-flight download, or remove a downloaded/failed entry from the
- * model cache (ROUTER mode only): DELETE /models?model=<repo:tag>.
+ * Cancel an in-flight download, or remove a downloaded/failed model from the
+ * cache (ROUTER mode only): DELETE /models?model=<repo:tag>. With a sidecar,
+ * only that sidecar of the cached model goes and the model stays.
*/
- static async cancelDownload(hfRepoWithTag: string): Promise<ApiModelsDownloadResponse> {
- return apiDelete<ApiModelsDownloadResponse>(API_MODELS.DELETE, {
- model: hfRepoWithTag
- });
+ static async cancelDownload(
+ model: string,
+ sidecar: ModelSidecar | null = null
+ ): Promise<ApiModelsDownloadResponse> {
+ return apiDelete<ApiModelsDownloadResponse>(
+ API_MODELS.DELETE,
+ sidecar ? { model, sidecar } : { model }
+ );
}
/**
* Start a model download from HuggingFace (ROUTER mode only). The response
* returns immediately; progress arrives over /models/sse. The server picks
- * the file matching the tag and also pulls the model's mmproj/draft sidecars.
+ * the model file matching the tag and the files of the sidecar kinds, as -hf
+ * does; posting a cached model again adds the sidecars it lacks.
*/
- static async downloadModel(hfRepoWithTag: string): Promise<ApiModelsDownloadResponse> {
- const payload: ApiModelsDownloadRequest = { model: hfRepoWithTag };
+ static async downloadModel(
+ model: string,
+ sidecars: ModelSidecar[]
+ ): Promise<ApiModelsDownloadResponse> {
+ const payload: ApiModelsDownloadRequest = { model, sidecars };
return apiPost<ApiModelsDownloadResponse>(API_MODELS.DOWNLOAD, payload);
}
+ /**
+ * Draft sidecar badges of the cached models, keyed by repo. Each listing entry
+ * reports the sidecar kinds it has on disk, never the file: the draft's own
+ * quant stays unknown here, and the `--model-draft` args report it once the
+ * model is loaded.
+ */
+ static draftSidecarsByRepo(
+ entries: { id: string; sidecars?: ModelSidecar[]; status?: { value?: unknown } }[]
+ ): Map<string, ModelSidecarBadge[]> {
+ const byRepo = new Map<string, ModelSidecarBadge[]>();
+
+ for (const { id, sidecars, status } of entries) {
+ // an unfinished download is not on disk yet; the download row tracks it
+ if (
+ status?.value === ServerModelStatus.DOWNLOADING ||
+ status?.value === ServerModelStatus.INCOMPLETE
+ )
+ continue;
+
+ const repo = repoOf(id);
+
+ for (const kind of sidecars ?? []) {
+ if (isAuxSidecar(kind)) continue;
+
+ const kinds = byRepo.get(repo) ?? [];
+
+ if (!kinds.some((b) => b.kind === kind && b.repo === repo)) {
+ kinds.push({ kind, quant: null, repo });
+ }
+
+ byRepo.set(repo, kinds);
+ }
+ }
+
+ return byRepo;
+ }
+
/**
* Check if a model is loaded based on its metadata.
*
@@ -74,10 +122,6 @@ export class ModelsService {
return model.status.value === ServerModelStatus.LOADED;
}
- static isModelLoading(model: ApiModelDataEntry): boolean {
- return model.status.value === ServerModelStatus.LOADING;
- }
-
/**
*
*
@@ -86,16 +130,8 @@ export class ModelsService {
*
*/
- /**
- * True when a router entry id marks a downloaded sidecar file, e.g.
- * `org/model:Q4_0-mtp` or `org/model:mmproj`, not a loadable model.
- */
- static isSidecarEntry(modelId: string): boolean {
- const idx = modelId.indexOf(MODEL_ID.QUANTIZATION_SEPARATOR);
-
- if (idx === MODEL_ID.NOT_FOUND) return false;
-
- return sidecarFromTag(modelId.slice(idx + 1)) !== null;
+ static isModelLoading(model: ApiModelDataEntry): boolean {
+ return model.status.value === ServerModelStatus.LOADING;
}
/**
@@ -191,8 +227,22 @@ export class ModelsService {
let modelPath: string;
if (colonIdx !== MODEL_ID.NOT_FOUND) {
- result.quantization = source.slice(colonIdx + 1) || null;
+ const tag = source.slice(colonIdx + 1);
+
modelPath = source.slice(0, colonIdx);
+
+ // a download tag carries the sidecar after the quant, e.g.
+ // `repo:Q8_0-dflash`, or a bare sidecar, e.g. `repo:mmproj`
+ const tagSuffix = tag.match(MODEL_ID.SIDECAR_SUFFIX_REGEX);
+
+ if (tagSuffix && MODEL_ID.QUANTIZATION_SEGMENT_REGEX.test(tagSuffix[1])) {
+ result.sidecar = sidecarFromFileToken(tagSuffix[2].toLowerCase());
+ result.quantization = tagSuffix[1];
+ } else if (sidecarFromFileToken(tag.toLowerCase())) {
+ result.sidecar = sidecarFromFileToken(tag.toLowerCase());
+ } else {
+ result.quantization = tag || null;
+ }
} else {
modelPath = source;
}
diff --git a/tools/ui/src/lib/stores/index.ts b/tools/ui/src/lib/stores/index.ts
index b57169907..fcab8a806 100644
--- a/tools/ui/src/lib/stores/index.ts
+++ b/tools/ui/src/lib/stores/index.ts
@@ -40,6 +40,9 @@ export { mcpStore } from './mcp/index.svelte';
// MODELS
export { modelsStore } from './models/index.svelte';
+// MODELS DISCOVER (HuggingFace browse state for the discover dialog)
+export { modelsDiscoverStore } from './models-discover/index.svelte';
+
// SERVER
export { serverStore } from './server.svelte';
diff --git a/tools/ui/src/lib/stores/models-discover/index.svelte.ts b/tools/ui/src/lib/stores/models-discover/index.svelte.ts
new file mode 100644
index 000000000..c4658485a
--- /dev/null
+++ b/tools/ui/src/lib/stores/models-discover/index.svelte.ts
@@ -0,0 +1,267 @@
+/**
+ * modelsDiscoverStore - Models Discover browse state
+ *
+ * Owns the HuggingFace GGUF list shown in the discover sidebar. The default list is
+ * the curated catalog set; a search replaces it with matches across all of HuggingFace.
+ * Both paths fetch the same fields, so a row renders the same either way.
+ */
+
+import { MODELS_DISCOVER_CATALOG_BATCH } from '$lib/constants';
+import { HuggingFaceService } from '$lib/services';
+import type {
+ HfCatalogBuild,
+ HfCatalogEntry,
+ HfModelInfo,
+ HfModelSibling
+} from '$lib/types/huggingface';
+import { isAuxSidecar } from '$lib/utils';
+import { SvelteMap } from 'svelte/reactivity';
+
+/** Min/max GGUF file size (bytes) across the quants of one repo. */
+export interface ModelsDiscoverSizeRange {
+ max: number;
+ min: number;
+}
+
+class ModelsDiscoverStore {
+ error = $state<string | null>(null);
+ models = $state<HfModelInfo[]>([]);
+ /** First model in the list - discover auto-opens this one. */
+ firstModel = $derived(this.models[0] ?? null);
+
+ loading = $state(false);
+ /** True while a search is in flight; only the newest request owns this flag. */
+ searching = $state(false);
+
+ private catalog: HfCatalogEntry[] = [];
+ /** Repo id -> size range, for catalog rows and lazily measured search rows. */
+ private catalogSizeRanges = new SvelteMap<string, ModelsDiscoverSizeRange>();
+ private defaultModels: HfModelInfo[] = [];
+ private fetched = false;
+ private searchRequestId = 0;
+ /** In-flight `sizeRange()` lookups, keyed by repo id. */
+ private sizeRangePending = new Map<string, Promise<ModelsDiscoverSizeRange | undefined>>();
+
+ /**
+ * Cached size range for a repo, without measuring: the synchronous part of
+ * `sizeRange()`, for rendering a row before its measurement resolves.
+ */
+ cachedSizeRangeFor(modelId: string): ModelsDiscoverSizeRange | undefined {
+ return this.catalogSizeRanges.get(modelId);
+ }
+
+ /** Catalog family description; undefined for a repo outside the curated catalog. */
+ descriptionFor(modelId: string): string | undefined {
+ return this.catalog.find((entry) =>
+ entry.sizes.some((size) => size.builds.some((build) => build.repo === modelId))
+ )?.description;
+ }
+
+ /**
+ * Fill the list from the llama.app catalog, one repo per catalog size. Every repo
+ * is fetched with its file tree, so a catalog row carries the same data a search
+ * result does. No-op when already loaded or in flight.
+ */
+ async fetch(): Promise<void> {
+ if (this.loading || this.fetched) return;
+
+ this.loading = true;
+ this.error = null;
+
+ try {
+ const catalog = await HuggingFaceService.getCatalog();
+
+ this.catalog = catalog;
+
+ const builds = this.catalogBuilds();
+ const failedBuilds: HfCatalogBuild[] = [];
+
+ this.defaultModels = [];
+
+ // fetch in small batches: not every repo hits the HF API at once,
+ // and the rows land as they arrive instead of all at the end
+ for (let i = 0; i < builds.length; i += MODELS_DISCOVER_CATALOG_BATCH) {
+ const batch = await Promise.all(
+ builds.slice(i, i + MODELS_DISCOVER_CATALOG_BATCH).map(async (build) => {
+ const [info, tree] = await Promise.all([
+ HuggingFaceService.getDetails(build.repo),
+ HuggingFaceService.getTree(build.repo)
+ ]);
+
+ return { build, info, tree };
+ })
+ );
+
+ for (const { build, info, tree } of batch) {
+ if (!info) {
+ failedBuilds.push(build);
+
+ continue;
+ }
+
+ this.catalogSizeRanges.set(build.repo, this.sizeRangeFor(build, tree));
+
+ // the catalog repo id, not the HF response `id`, drives selection
+ this.defaultModels.push({ ...info, id: build.repo, modelId: build.repo });
+ }
+
+ // show what has landed, unless a search owns the list right now
+ if (!this.searching) {
+ this.models = [...this.defaultModels];
+ }
+ }
+
+ // stay incomplete when some repos failed, so the next mount retries
+ // them; succeeded repos are served from the service cache for free
+ this.fetched = failedBuilds.length === 0;
+ } catch (err) {
+ this.error = err instanceof Error ? err.message : 'Failed to fetch models';
+ } finally {
+ this.loading = false;
+ }
+ }
+
+ /** Replace the list with search results; an empty query restores the default list. */
+ async search(query: string): Promise<void> {
+ const trimmed = query.trim();
+ const requestId = ++this.searchRequestId;
+
+ if (!trimmed) {
+ this.searching = false;
+ this.models = this.defaultModels;
+ this.error = null;
+
+ return;
+ }
+
+ this.searching = true;
+
+ try {
+ const results = await HuggingFaceService.searchByQuery(trimmed, { limit: 50 });
+
+ if (requestId !== this.searchRequestId) return;
+
+ this.models = results;
+ this.error = null;
+ } catch (err) {
+ if (requestId !== this.searchRequestId) return;
+
+ this.error = err instanceof Error ? err.message : 'Search failed';
+ } finally {
+ if (requestId === this.searchRequestId) this.searching = false;
+ }
+ }
+
+ /**
+ * Size range for a repo not measured yet, fetched once and cached: a search result
+ * never goes through fetch(), and a row can be remounted while scrolling.
+ */
+ sizeRange(modelId: string): Promise<ModelsDiscoverSizeRange | undefined> {
+ const cached = this.catalogSizeRanges.get(modelId);
+
+ if (cached) return Promise.resolve(cached);
+
+ const pending = this.sizeRangePending.get(modelId);
+
+ if (pending) return pending;
+
+ const request = (async () => {
+ const tree = await HuggingFaceService.getTree(modelId);
+ const range = this.sizeRangeOfMainQuants(tree);
+
+ if (range) this.catalogSizeRanges.set(modelId, range);
+
+ return range;
+ })()
+ .catch(() => undefined)
+ .finally(() => this.sizeRangePending.delete(modelId));
+
+ this.sizeRangePending.set(modelId, request);
+
+ return request;
+ }
+
+ /**
+ * Sizes the catalog lists under this repo. Never empty: an entry with no parsable
+ * size contributes the build's own.
+ */
+ private buildSizeBytes(build: HfCatalogBuild): number[] {
+ const sizes = this.catalog
+ .flatMap((entry) => entry.sizes)
+ .flatMap((size) => size.builds.filter((b) => b.repo === build.repo))
+ .map((b) => b.sizeBytes ?? HuggingFaceService.parseSizeBytes(b.size))
+ .filter((bytes): bytes is number => Boolean(bytes) && bytes > 0);
+
+ return sizes.length > 0 ? sizes : [build.sizeBytes ?? 0];
+ }
+
+ /**
+ * One build per catalog size, newest family first. Prefers the official ggml-org
+ * repo, else the first build, so repos of other orgs are covered too.
+ */
+ private catalogBuilds(): HfCatalogBuild[] {
+ return [...this.catalog]
+ .sort((a, b) => b.released.localeCompare(a.released))
+ .flatMap((entry) =>
+ entry.sizes.flatMap((size) => {
+ const build = size.builds.find((b) => b.repo.startsWith('ggml-org/')) ?? size.builds[0];
+
+ return build ? [build] : [];
+ })
+ );
+ }
+
+ /** Byte sizes of every non-sidecar quant file in a tree, shards collapsed. */
+ private quantSizesOf(tree: HfModelSibling[]): number[] {
+ return HuggingFaceService.collapseGgufShards(
+ HuggingFaceService.filterByExtension(tree, '.gguf')
+ )
+ .filter((f) => {
+ const { quant, sidecar } = HuggingFaceService.extractQuantMeta(f.path) ?? {};
+
+ return Boolean(quant) && (sidecar === null || sidecar === undefined);
+ })
+ .map((f) => f.size ?? 0)
+ .filter((size) => size > 0);
+ }
+
+ /**
+ * Size range of one catalog build, draft sidecars included so the range covers what
+ * gets downloaded. Falls back to the catalog's own sizes when the tree yielded nothing.
+ */
+ private sizeRangeFor(build: HfCatalogBuild, tree: HfModelSibling[]): ModelsDiscoverSizeRange {
+ const quantSizes = this.quantSizesOf(tree);
+
+ if (quantSizes.length === 0) {
+ const listed = this.buildSizeBytes(build);
+
+ return { max: Math.max(...listed), min: Math.min(...listed) };
+ }
+
+ const draftSizes = tree
+ .filter((f) => {
+ const sidecar = HuggingFaceService.extractQuantMeta(f.path)?.sidecar;
+
+ return sidecar !== null && sidecar !== undefined && !isAuxSidecar(sidecar);
+ })
+ .map((f) => f.size ?? 0)
+ .filter((size) => size > 0);
+ const extra = draftSizes.length > 0 ? Math.max(...draftSizes) : 0;
+
+ return {
+ max: Math.max(...quantSizes) + extra,
+ min: Math.min(...quantSizes) + (draftSizes.length > 0 ? Math.min(...draftSizes) : 0)
+ };
+ }
+
+ /** Size range across the main-model quants of a tree, draft sidecars excluded. */
+ private sizeRangeOfMainQuants(tree: HfModelSibling[]): ModelsDiscoverSizeRange | undefined {
+ const sizes = this.quantSizesOf(tree);
+
+ if (sizes.length === 0) return undefined;
+
+ return { max: Math.max(...sizes), min: Math.min(...sizes) };
+ }
+}
+
+export const modelsDiscoverStore = new ModelsDiscoverStore();
diff --git a/tools/ui/src/lib/stores/models/index.svelte.ts b/tools/ui/src/lib/stores/models/index.svelte.ts
index a6b081b9c..38d734f4e 100644
--- a/tools/ui/src/lib/stores/models/index.svelte.ts
+++ b/tools/ui/src/lib/stores/models/index.svelte.ts
@@ -23,10 +23,9 @@ import { conversationsStore } from '$lib/stores/conversations/index.svelte';
import { type ModelPropsHost, ModelPropsManager } from '$lib/stores/models/props.svelte';
import { type ModelStatusHost, ModelStatusManager } from '$lib/stores/models/status.svelte';
import { serverStore } from '$lib/stores/server.svelte';
-import type { ModelSidecarBadge, ParsedModelId } from '$lib/types/models';
+import type { ParsedModelId } from '$lib/types/models';
import { getConversationModel } from '$lib/utils/conversation-utils';
import { repoOf } from '$lib/utils/model-names';
-import { isAuxSidecar } from '$lib/utils/sidecars';
import { SvelteMap, SvelteSet } from 'svelte/reactivity';
import { toast } from 'svelte-sonner';
@@ -39,24 +38,6 @@ const MAX_GROUP_OPEN_ENTRIES = 200;
*/
const HUB_WARM_LIMIT = 12;
-/** Union of the draft sidecar badges two sources report for one repo. */
-function mergedDraftSidecars(
- fromListing: ModelSidecarBadge[] | undefined,
- fromArgs: ModelSidecarBadge[]
-): ModelSidecarBadge[] {
- if (!fromListing || fromListing.length === 0) return fromArgs;
-
- if (fromArgs.length === 0) return fromListing;
-
- const merged = [...fromListing];
-
- for (const badge of fromArgs) {
- if (!merged.some((b) => b.kind === badge.kind && b.repo === badge.repo)) merged.push(badge);
- }
-
- return merged;
-}
-
/** Models kept out of the selector. */
function loadHiddenModels(): Set<string> {
if (!browser) return new SvelteSet<string>();
@@ -307,6 +288,9 @@ class ModelsStore implements ModelPropsHost, ModelStatusHost {
const response = await ModelsService.list();
this.routerModels = response.data;
+ // a download an earlier run left unfinished is listed as incomplete, and
+ // is offered as paused until it is resumed
+ this._status.syncPausedDownloads();
// keep the selector options in sync: a downloaded / deleted model shows
// up here too, not only in the router model rows
this.models = this.buildModelOptions(response);
@@ -576,40 +560,19 @@ class ModelsStore implements ModelPropsHost, ModelStatusHost {
item,
parsed: ModelsService.parseModelId(item.id)
}));
- // sidecar entries mark downloaded sidecar files, not loadable models, and the router
- // lists them under their own `<repo>:<quant>-<sidecar>` id: pair each draft with its
- // repo here and drop the entries below
- const draftSidecarsByRepo = new SvelteMap<string, ModelSidecarBadge[]>();
-
- for (const { item, parsed } of entries) {
- if (!parsed.sidecar || isAuxSidecar(parsed.sidecar)) continue;
-
- const repo = repoOf(item.id);
- // the entry id's quant is the sidecar file's own quant
- const badge: ModelSidecarBadge = {
- kind: parsed.sidecar,
- quant: parsed.quantization,
- repo
- };
- const kinds = draftSidecarsByRepo.get(repo);
-
- if (kinds) {
- if (!kinds.some((b) => b.kind === badge.kind && b.repo === badge.repo)) {
- kinds.push(badge);
- }
- } else {
- draftSidecarsByRepo.set(repo, [badge]);
- }
- }
return (
entries
- // the router also lists projector files by filename, which carry the
- // sidecar token inside the id rather than after a colon
- .filter(({ item, parsed }) => !parsed.sidecar && !ModelsService.isSidecarEntry(item.id))
- // in-flight downloads are not usable models yet; the selector tracks
- // them in its "Download in progress" section instead
- .filter(({ item }) => item.status?.value !== ServerModelStatus.DOWNLOADING)
+ // a models dir can still name an entry after a projector file, which is
+ // no loadable model
+ .filter(({ parsed }) => !parsed.sidecar)
+ // in-flight and unfinished downloads are not usable models yet; the
+ // selector tracks them in its "Download in progress" section instead
+ .filter(
+ ({ item }) =>
+ item.status?.value !== ServerModelStatus.DOWNLOADING &&
+ item.status?.value !== ServerModelStatus.INCOMPLETE
+ )
.map(({ details, item }) => {
const rawCapabilities = Array.isArray(details?.capabilities) ? details?.capabilities : [];
const displayNameSource =
@@ -631,12 +594,7 @@ class ModelsStore implements ModelPropsHost, ModelStatusHost {
: undefined),
description: details?.description,
details: details?.details,
- draftSidecars: mergedDraftSidecars(
- draftSidecarsByRepo.get(repo),
- // models the router serves with a draft record it in their
- // --model-draft args instead of a listing entry
- this._status.getDraftSidecars(repo)
- ),
+ draftSidecars: this._status.getDraftSidecars(repo),
id: item.id,
meta: item.meta ?? null,
modalities: this.props.buildArchitectureModalities(item.architecture),
diff --git a/tools/ui/src/lib/stores/models/status.svelte.ts b/tools/ui/src/lib/stores/models/status.svelte.ts
index fc2d52b12..4c00adbd7 100644
--- a/tools/ui/src/lib/stores/models/status.svelte.ts
+++ b/tools/ui/src/lib/stores/models/status.svelte.ts
@@ -5,14 +5,20 @@
* Created and owned by modelsStore, which also owns the router rows it updates.
*/
+import { browser } from '$app/environment';
import {
CLI_FLAGS,
HF_UD_QUANT_PREFIX_REGEX,
MODEL_ID,
- PATH_SEPARATOR,
- PAUSED_MODEL_DOWNLOADS_LOCALSTORAGE_KEY
+ type ModelSidecar,
+ STORAGE_APP_NAME
} from '$lib/constants';
-import { ModelDownloadStopRequest, ServerModelsSseEventType, ServerModelStatus } from '$lib/enums';
+import {
+ ModelAuxSidecar,
+ ModelDownloadStopRequest,
+ ServerModelsSseEventType,
+ ServerModelStatus
+} from '$lib/enums';
import { HuggingFaceService } from '$lib/services/huggingface.service';
import { ModelsService } from '$lib/services/models.service';
import type { ModelPropsManager } from '$lib/stores/models/props.svelte';
@@ -26,7 +32,7 @@ import type {
} from '$lib/types';
import type { ModelSidecarBadge } from '$lib/types/models';
import { repoOf } from '$lib/utils/model-names';
-import { isAuxSidecar } from '$lib/utils/sidecars';
+import { isAuxSidecar, isDraftSidecar, sidecarFromFileToken } from '$lib/utils/sidecars';
import { SvelteMap, SvelteSet } from 'svelte/reactivity';
import { toast } from 'svelte-sonner';
@@ -59,40 +65,37 @@ function downloadIdKey(repoWithTag: string): string {
return `${repo.toUpperCase()}:${tag.toUpperCase().replace(HF_UD_QUANT_PREFIX_REGEX, '')}`;
}
-export class ModelStatusManager {
- /**
- * Sidecar files pulled by registered models, as `<repo>/<file>` keys.
- * Sidecars are not separate /v1/models entries - the router pulls them as
- * sidecars of a main model and records them in its `--model-draft` /
- * `--mmproj` args.
- */
- private downloadedSidecars = $derived.by(() => {
- const result = new SvelteSet<string>();
-
- for (const m of this.host.routerModels) {
- const args = m.status?.args;
+/** Sidecar kinds the server pulls with a model, as -hf does. */
+const SERVER_SIDECARS: readonly string[] = ['mmproj', 'mtp', 'dflash', 'eagle3', 'dspark'];
- if (!args) continue;
+/**
+ * The model a download option stands for, and the sidecar it adds: `<repo>:<quant>`
+ * is the model itself, `<repo>:<quant>-<sidecar>` a sidecar of that model, and a
+ * sidecar without quant belongs to the repo's default model.
+ */
+function downloadTarget(repoWithTag: string): { model: string; sidecar: ModelSidecar | null } {
+ const parsed = ModelsService.parseModelId(repoWithTag);
- for (let i = 0; i < args.length - 1; i++) {
- if (
- args[i] !== CLI_FLAGS.MODEL_DRAFT &&
- args[i] !== CLI_FLAGS.MODEL_DRAFT_SHORT &&
- args[i] !== CLI_FLAGS.MMPROJ
- ) {
- continue;
- }
+ if (!parsed.sidecar) return { model: repoWithTag, sidecar: null };
- const parsed = HuggingFaceService.parseCachePath(args[i + 1]);
+ const repo = repoOf(repoWithTag);
+ const model = parsed.quantization
+ ? `${repo}${MODEL_ID.QUANTIZATION_SEPARATOR}${parsed.quantization}`
+ : repo;
- if (parsed) result.add(`${parsed.repo}${PATH_SEPARATOR}${parsed.file}`);
- }
- }
+ return { model, sidecar: parsed.sidecar };
+}
- return result;
- });
+/** Zeroed progress, used until the first feed record reports real counts. */
+function emptyDownloadProgress(): ModelDownloadProgress {
+ return { downloadedBytes: 0, files: {}, totalBytes: 0 };
+}
+export class ModelStatusManager {
+ /** In-flight downloads by model, with their aggregated progress. */
private downloadProgress = new SvelteMap<string, ModelDownloadProgress>();
+ /** Sidecar kinds each in-flight or paused download pulls with its model. */
+ private downloadSidecars = new SvelteMap<string, ModelSidecar[]>();
/**
* Draft sidecars the registered models pull with them, keyed by the repo of the model
* that pulls them: the router records them in the model's `--model-draft` args.
@@ -128,7 +131,11 @@ export class ModelStatusManager {
const kinds = result.get(baseRepo);
if (kinds) {
- if (!kinds.some((b) => b.kind === badge.kind && b.repo === badge.repo)) {
+ if (
+ !kinds.some(
+ (b) => b.kind === badge.kind && b.repo === badge.repo && b.quant === badge.quant
+ )
+ ) {
kinds.push(badge);
}
} else {
@@ -139,27 +146,71 @@ export class ModelStatusManager {
return result;
});
- /** `<repo>:<tag>` strings whose most recent download attempt failed (download_failed). */
+ /** Models whose most recent download attempt failed (download_failed). */
private failedDownloads = new SvelteSet<string>();
+ /** Draft sidecars the cached models report on disk, keyed by repo. */
+ private listedSidecarsByRepo = $derived.by(() =>
+ ModelsService.draftSidecarsByRepo(this.host.routerModels)
+ );
private loadingStates = new SvelteMap<string, boolean>();
private loadProgress = new SvelteMap<string, ModelLoadProgress>();
/** Paused downloads with their last reported progress, or null when none arrived before the pause. */
private pausedDownloads = new SvelteMap<string, ModelDownloadProgress | null>();
+ /** Models with a download POST in flight, so a second click cannot post again. */
+ private pendingDownloads = new SvelteSet<string>();
+ /**
+ * Draft kinds a download on its way or unfinished reports, keyed by repo: the
+ * files are not on disk yet, so a download row reads them as pending.
+ */
+ private pendingSidecarsByRepo = $derived.by(() => {
+ const result = new SvelteMap<string, ModelSidecarBadge[]>();
+
+ for (const m of this.host.routerModels) {
+ if (
+ m.status?.value !== ServerModelStatus.DOWNLOADING &&
+ m.status?.value !== ServerModelStatus.INCOMPLETE
+ )
+ continue;
+
+ if (!Array.isArray(m.sidecars)) continue;
+
+ const repo = repoOf(m.id);
+
+ for (const kind of m.sidecars) {
+ const sidecar = sidecarFromFileToken(kind);
+
+ if (!sidecar || !isDraftSidecar(sidecar)) continue;
+
+ const badge: ModelSidecarBadge = { kind: sidecar, pending: true, quant: null, repo };
+ const badges = result.get(repo);
+
+ if (badges) {
+ if (!badges.some((b) => b.kind === badge.kind && b.repo === badge.repo)) {
+ badges.push(badge);
+ }
+ } else {
+ result.set(repo, [badge]);
+ }
+ }
+ }
+
+ return result;
+ });
// /models/sse feed state, the single source of truth for status and load progress
private statusAbort: AbortController | null = null;
private statusReaderActive = false;
+
private statusWaiters = new SvelteMap<
string,
{ target: ServerModelStatus; resolve: () => void; reject: (e: Error) => void }
>();
- /** Tags the user asked to stop (pause or cancel); the download_failed the stop triggers is intentional, not a failure. */
+ /** Models the user asked to stop (pause or cancel); the download_failed the stop triggers is intentional, not a failure. */
private stopRequests = new SvelteMap<string, ModelDownloadStopRequest>();
/**
- * Cancel an in-flight download, or remove a downloaded/failed entry from the
- * cache (ROUTER mode only). The server's remove endpoint covers both: dropping
- * a running download discards its partial files, dropping a cached model
- * deletes the files on disk, so the delete action resolves through this too.
+ * Cancel an in-flight download, or remove a downloaded/failed model from the cache
+ * (ROUTER mode only). A sidecar option removes that sidecar alone; anything else
+ * removes the model along with the partial files of its download.
*/
async cancelDownload(repoWithTag: string): Promise<boolean> {
if (!serverStore.isRouterMode) {
@@ -170,29 +221,37 @@ export class ModelStatusManager {
this.subscribe();
+ const { model, sidecar } = downloadTarget(repoWithTag);
// in-flight: the kill triggers download_failed over the feed; mark it as a
// user cancel so it settles silently instead of toasting a failure
- if (this.downloadProgress.has(repoWithTag)) {
- this.stopRequests.set(repoWithTag, ModelDownloadStopRequest.CANCEL);
+ const wasInFlight = this.downloadProgress.has(model);
+
+ if (wasInFlight) {
+ this.stopRequests.set(model, ModelDownloadStopRequest.CANCEL);
}
// a downloaded model registers under the name the router derived from the
// cached file (e.g. the UD- quant prefix is dropped), so resolve the tag to
// the registered id before asking the server to remove it
const registeredId =
- this.host.routerModels.find((m) => downloadIdKey(m.id) === downloadIdKey(repoWithTag))?.id ??
- repoWithTag;
+ this.host.routerModels.find((m) => downloadIdKey(m.id) === downloadIdKey(model))?.id ?? model;
+ const sidecarOnly = sidecar !== null && !wasInFlight && this.isModelDownloaded(model);
try {
- const res = await ModelsService.cancelDownload(registeredId);
+ const res = await ModelsService.cancelDownload(registeredId, sidecarOnly ? sidecar : null);
const ok = res.success === true;
- if (ok) {
- this.downloadProgress.delete(repoWithTag);
- this.failedDownloads.delete(repoWithTag);
- this.deletePausedDownload(repoWithTag);
+ if (ok && !sidecarOnly) {
+ this.downloadProgress.delete(model);
+ this.downloadSidecars.delete(model);
+ this.failedDownloads.delete(model);
+ this.deletePausedDownload(model);
}
+ // the feed's model_remove refreshes the list too, but refetch here as well:
+ // a missed event, or a sidecar removal that sends none, would leave it stale
+ if (ok) void this.host.fetchRouterModels();
+
return ok;
} catch (error) {
toast.error(`Failed to cancel: ${error instanceof Error ? error.message : 'unknown error'}`);
@@ -221,58 +280,93 @@ export class ModelStatusManager {
}
constructor(private host: ModelStatusHost) {
- // the server has no notion of a paused download, so the ids survive in
- // localStorage; the progress snapshot is stale after a reload and stays null
- try {
- const raw = localStorage.getItem(PAUSED_MODEL_DOWNLOADS_LOCALSTORAGE_KEY);
- const parsed: unknown = JSON.parse(raw ?? '[]');
-
- if (!Array.isArray(parsed)) return;
-
- for (const repoWithTag of parsed.filter((id): id is string => typeof id === 'string')) {
- this.pausedDownloads.set(repoWithTag, null);
- }
- } catch {
- // unreadable or corrupt: start without the paused set
- }
+ // the paused-ids registry moved to the server's unfinished-downloads record;
+ // drop the key this store used to persist
+ if (browser) localStorage.removeItem(`${STORAGE_APP_NAME}.pausedModelDownloads`);
}
/**
- * POST /models starts the download in the background; the feed reports
- * progress, and models_reload refreshes the list once it finishes.
- * Re-posting a paused tag resumes from the partial files kept on disk.
+ * POST /models starts the download in the background; the feed reports progress and
+ * models_reload refreshes the list once it finishes. A sidecar option posts its model
+ * with that sidecar on top of the ones it already pulls, so the server keeps what is
+ * on disk and adds the rest; re-posting a paused model resumes it.
*/
- async downloadModel(repoWithTag: string): Promise<void> {
+ async downloadModel(repoWithTag: string, extraSidecars: ModelSidecar[] = []): Promise<void> {
if (!serverStore.isRouterMode) {
toast.error('Model downloads are only available in router mode');
return;
}
+ const { model, sidecar } = downloadTarget(repoWithTag);
+
+ if (sidecar && !SERVER_SIDECARS.includes(sidecar)) {
+ toast.error(`The ${sidecar} file is not downloadable from here`);
+
+ return;
+ }
+
+ const registered = this.host.routerModels.find(
+ (m) => downloadIdKey(m.id) === downloadIdKey(model)
+ );
+ const sidecars = [
+ ModelAuxSidecar.MMPROJ,
+ ...(registered?.sidecars ?? []),
+ ...(this.downloadSidecars.get(model) ?? []),
+ ...(sidecar ? [sidecar] : []),
+ ...extraSidecars
+ ].filter((kind, i, all) => SERVER_SIDECARS.includes(kind) && all.indexOf(kind) === i);
+
+ // a second click must not post again: the server would answer that the model
+ // already exists, so a download already on its way is left alone
+ if (this.pendingDownloads.has(model) || this.isDownloadInProgress(model)) return;
+
+ this.pendingDownloads.add(model);
+
// the feed must be live so the resulting models_reload event refreshes the list
this.subscribe();
- // resuming a paused download: drop the paused state, and let the server
- // discard its stale DOWNLOADED entry (via the list fetch) before re-posting
- if (this.deletePausedDownload(repoWithTag) || this.stopRequests.delete(repoWithTag)) {
- await this.host.fetchRouterModels();
- }
+ // the marker and its last reported progress survive until the request is
+ // accepted: dropping them first would drop the paused state, which reads as
+ // a lost download if the server rejects the resume
+ const snapshot = this.pausedDownloads.get(model) ?? null;
+ const wasPaused = this.pausedDownloads.has(model);
+
+ // the download reads as in flight from here, so the chip shows it before the
+ // server answers; the feed refines it with the real counts
+ this.downloadProgress.set(model, snapshot ?? emptyDownloadProgress());
try {
- const res = await ModelsService.downloadModel(repoWithTag);
+ const res = await ModelsService.downloadModel(model, sidecars);
if (!res.success) {
throw new Error(res.error?.message ?? 'Server rejected the download request');
}
- // flip the chip to "downloading" right away; the feed refines it with real progress
- this.downloadProgress.set(repoWithTag, { downloadedBytes: 0, files: {}, totalBytes: 0 });
+ // the marker gives way to the live entry, seeded with the last reported
+ // progress so the row does not fall back to 0%
+ this.deletePausedDownload(model);
+ this.stopRequests.delete(model);
+ this.failedDownloads.delete(model);
+ this.downloadSidecars.set(model, sidecars);
- toast.success(`Download started: ${this.host.toDisplayName(repoWithTag)}`);
+ toast.success(`Download started: ${this.host.toDisplayName(model)}`);
} catch (error) {
- toast.error(`Download failed: ${repoWithTag}`);
+ // the download did not start: the optimistic entry goes, and a resumed one
+ // stays paused rather than dropping off the list
+ this.downloadProgress.delete(model);
+
+ if (wasPaused) {
+ this.setPausedDownload(model, snapshot);
+ }
+
+ const reason = error instanceof Error ? error.message : 'unknown error';
+
+ toast.error(`Download failed: ${model} (${reason})`);
throw error;
+ } finally {
+ this.pendingDownloads.delete(model);
}
}
@@ -283,31 +377,51 @@ export class ModelStatusManager {
}
/**
- * Tracked downloads (in flight or paused) for the selector's
- * "Download in progress" section.
+ * Tracked downloads (in flight or paused) for the selector's "Download in
+ * progress" section: one entry per model, its sidecars riding with it.
*/
getDownloadEntries(): ModelDownloadEntry[] {
- const inFlight = Array.from(this.downloadProgress, ([repoWithTag, progress]) => ({
+ const inFlight = Array.from(this.downloadProgress, ([model, progress]) => ({
isPaused: false,
progress,
- repoWithTag
+ repoWithTag: model
}));
- const paused = Array.from(this.pausedDownloads, ([repoWithTag, progress]) => ({
+ // a model can carry a stale pause mark next to its live entry, e.g. after
+ // the list adopted it again while it was resuming: one row, the live one
+ const paused = Array.from(this.pausedDownloads, ([model, progress]) => ({
isPaused: true,
progress,
- repoWithTag
- }));
+ repoWithTag: model
+ })).filter((entry) => !this.downloadProgress.has(entry.repoWithTag));
return [...inFlight, ...paused];
}
getDownloadProgress(repoWithTag: string): ModelDownloadProgress | null {
- return this.downloadProgress.get(repoWithTag) ?? null;
+ return this.isDownloadInProgress(repoWithTag)
+ ? (this.downloadProgress.get(downloadTarget(repoWithTag).model) ?? null)
+ : null;
}
- /** Draft sidecars a repo pulls with its model, e.g. `mtp` at `Q8_0`. */
+ /**
+ * Draft sidecars a repo pulls with its model, e.g. `mtp` at `Q8_0`: the
+ * router records loaded models' sidecars in their `--model-draft` args, and
+ * lists the sidecars each cached model has on disk.
+ */
getDraftSidecars(repoId: string): ModelSidecarBadge[] {
- return this.draftSidecarsByRepo.get(repoId) ?? [];
+ // one badge per kind: the args know the quant, the listing carries kinds only,
+ // and a draft still coming down is pending rather than on disk
+ const merged = [...(this.draftSidecarsByRepo.get(repoId) ?? [])];
+
+ for (const source of [this.listedSidecarsByRepo, this.pendingSidecarsByRepo]) {
+ for (const badge of source.get(repoId) ?? []) {
+ if (!merged.some((b) => b.kind === badge.kind && b.repo === badge.repo)) {
+ merged.push(badge);
+ }
+ }
+ }
+
+ return merged;
}
getLoadProgress(modelId: string): ModelLoadProgress | null {
@@ -315,40 +429,59 @@ export class ModelStatusManager {
}
getPausedDownloadProgress(repoWithTag: string): ModelDownloadProgress | null {
- return this.pausedDownloads.get(repoWithTag) ?? null;
+ return this.isDownloadPaused(repoWithTag)
+ ? (this.pausedDownloads.get(downloadTarget(repoWithTag).model) ?? null)
+ : null;
}
hasFailedDownload(repoWithTag: string): boolean {
- return this.failedDownloads.has(repoWithTag);
+ const { model } = downloadTarget(repoWithTag);
+
+ return this.failedDownloads.has(model) && this.pullsSidecar(repoWithTag);
}
- /** Active while the feed reports download_progress for the tag. */
+ /** Active while the feed reports download_progress for the option's model, and it pulls the option. */
isDownloadInProgress(repoWithTag: string): boolean {
- return this.downloadProgress.has(repoWithTag);
+ return (
+ this.downloadProgress.has(downloadTarget(repoWithTag).model) && this.pullsSidecar(repoWithTag)
+ );
}
isDownloadPaused(repoWithTag: string): boolean {
- return this.pausedDownloads.has(repoWithTag);
+ const { model } = downloadTarget(repoWithTag);
+
+ // a running download outranks its pause mark, which can outlive a resume
+ return (
+ this.pausedDownloads.has(model) &&
+ !this.downloadProgress.has(model) &&
+ this.pullsSidecar(repoWithTag)
+ );
}
/**
- * True when the tag is already registered in the /v1/models list; both ids
- * are normalized, see downloadIdKey().
+ * True when the option's model is registered in the /v1/models list and, for
+ * a sidecar option, reports that sidecar on disk; both ids are normalized,
+ * see downloadIdKey().
*/
isModelDownloaded(repoWithTag: string): boolean {
- const key = downloadIdKey(repoWithTag);
+ const { model, sidecar } = downloadTarget(repoWithTag);
+ const key = downloadIdKey(model);
- return this.host.routerModels.some((m) => downloadIdKey(m.id) === key);
+ // an unfinished download (downloading or incomplete) is not a model on disk,
+ // so the download options offer it as resumable instead
+ return this.host.routerModels.some(
+ (m) =>
+ downloadIdKey(m.id) === key &&
+ m.status?.value !== ServerModelStatus.DOWNLOADING &&
+ m.status?.value !== ServerModelStatus.INCOMPLETE &&
+ (!sidecar || (m.sidecars ?? []).includes(sidecar))
+ );
}
isOperationInProgress(modelId: string): boolean {
return this.loadingStates.get(modelId) ?? false;
}
- isSidecarDownloaded(repoId: string, filePath: string): boolean {
- return this.downloadedSidecars.has(`${repoId}/${filePath}`);
- }
-
async load(modelId: string): Promise<void> {
if (this.host.isModelLoaded(modelId)) return;
@@ -380,9 +513,10 @@ export class ModelStatusManager {
}
/**
- * The server stops the download child but keeps the partial files, so
- * re-posting the tag resumes where it stopped. The feed reports the stop
- * as download_failed; the 'pause' stop request marks it as intentional.
+ * The server stops the download child but keeps the partial files, so re-posting
+ * the model resumes where it stopped. The feed reports the stop as download_failed;
+ * the 'pause' stop request marks it as intentional, and the paused state is stored
+ * before the request: an unconfirmed pause would leave the chip spinning.
*/
async pauseDownload(repoWithTag: string): Promise<void> {
if (!serverStore.isRouterMode) {
@@ -391,15 +525,32 @@ export class ModelStatusManager {
return;
}
+ const { model } = downloadTarget(repoWithTag);
+
+ // a POST still on its way has nothing to stop: unloading would kill a model
+ // that never started and leave the store neither in flight nor paused
+ if (this.pendingDownloads.has(model)) return;
+
this.subscribe();
- this.stopRequests.set(repoWithTag, ModelDownloadStopRequest.PAUSE);
+ this.stopRequests.set(model, ModelDownloadStopRequest.PAUSE);
+
+ const snapshot = this.downloadProgress.get(model) ?? null;
+ const wasInFlight = this.downloadProgress.delete(model);
+
+ this.setPausedDownload(model, snapshot);
try {
- await ModelsService.unload(repoWithTag);
+ await ModelsService.unload(model);
} catch {
- this.stopRequests.delete(repoWithTag);
- toast.error(`Failed to pause: ${repoWithTag}`);
+ this.stopRequests.delete(model);
+ this.deletePausedDownload(model);
+
+ if (wasInFlight) {
+ this.downloadProgress.set(model, snapshot ?? emptyDownloadProgress());
+ }
+
+ toast.error(`Failed to pause: ${model}`);
}
}
@@ -414,6 +565,36 @@ export class ModelStatusManager {
void this.runStatusReader(this.statusAbort.signal);
}
+ /**
+ * A model the server lists as incomplete is a download that never finished: it
+ * runs in no tab and carries no progress. Offer it as paused with the sidecars
+ * it pulls, and re-posting the model resumes it from the files already on disk.
+ */
+ syncPausedDownloads(): void {
+ for (const model of this.host.routerModels) {
+ if (model.status?.value !== ServerModelStatus.INCOMPLETE) continue;
+
+ if (!this.downloadSidecars.has(model.id)) {
+ this.downloadSidecars.set(model.id, model.sidecars ?? []);
+ }
+
+ if (this.pausedDownloads.has(model.id)) continue;
+
+ this.setPausedDownload(model.id, null);
+ }
+ }
+
+ /**
+ * Pause an in-flight download, or resume a paused or incomplete one: one
+ * download per model, its sidecars riding along with it.
+ */
+ async toggleDownload(repoWithTag: string): Promise<void> {
+ const { model } = downloadTarget(repoWithTag);
+
+ if (this.isDownloadPaused(model)) await this.downloadModel(model);
+ else await this.pauseDownload(model);
+ }
+
async unload(modelId: string): Promise<void> {
if (!this.host.isModelLoaded(modelId)) return;
@@ -473,12 +654,17 @@ export class ModelStatusManager {
// user cancel: settle silently, the feed's model_remove cleans up the entry
this.failedDownloads.delete(event.model);
this.deletePausedDownload(event.model);
+ this.downloadSidecars.delete(event.model);
return;
}
if (request === ModelDownloadStopRequest.PAUSE) {
- this.setPausedDownload(event.model, progress);
+ // a pause already stored its snapshot when the user asked for it
+ if (!this.pausedDownloads.has(event.model)) {
+ this.setPausedDownload(event.model, progress);
+ }
+
this.failedDownloads.delete(event.model);
return;
@@ -490,6 +676,7 @@ export class ModelStatusManager {
if (ok) {
this.failedDownloads.delete(event.model);
+ this.downloadSidecars.delete(event.model);
// the finished download only registers in /v1/models on the next list
// fetch (the server reloads its model table then), so refetch to flip
@@ -589,22 +776,14 @@ export class ModelStatusManager {
}
private deletePausedDownload(repoWithTag: string): boolean {
- if (!this.pausedDownloads.delete(repoWithTag)) return false;
-
- this.persistPausedDownloads();
-
- return true;
+ return this.pausedDownloads.delete(repoWithTag);
}
- private persistPausedDownloads(): void {
- try {
- localStorage.setItem(
- PAUSED_MODEL_DOWNLOADS_LOCALSTORAGE_KEY,
- JSON.stringify(Array.from(this.pausedDownloads.keys()))
- );
- } catch {
- // storage unavailable: the pauses just do not survive a reload
- }
+ /** True when the option is the model itself, or a sidecar its download pulls. */
+ private pullsSidecar(repoWithTag: string): boolean {
+ const { model, sidecar } = downloadTarget(repoWithTag);
+
+ return !sidecar || (this.downloadSidecars.get(model) ?? []).includes(sidecar);
}
private rejectStatus(modelId: string, error: Error): void {
@@ -617,18 +796,21 @@ export class ModelStatusManager {
}
private removeRouterModel(modelId: string): void {
- if (this.host.routerModels.findIndex((m) => m.id === modelId) === -1) return;
-
- this.host.routerModels = this.host.routerModels.filter((m) => m.id !== modelId);
- this.loadProgress.delete(modelId);
- this.downloadProgress.delete(modelId);
- this.failedDownloads.delete(modelId);
- this.deletePausedDownload(modelId);
- this.stopRequests.delete(modelId);
- this.rejectStatus(modelId, new Error(`Model removed: ${this.host.toDisplayName(modelId)}`));
-
- // drop the row from the selector options too; they rebuild from the list
- // response, which only a refetch provides
+ const idx = this.host.routerModels.findIndex((m) => m.id === modelId);
+
+ if (idx !== -1) {
+ this.host.routerModels = this.host.routerModels.filter((m) => m.id !== modelId);
+ this.loadProgress.delete(modelId);
+ this.downloadProgress.delete(modelId);
+ this.failedDownloads.delete(modelId);
+ this.deletePausedDownload(modelId);
+ this.downloadSidecars.delete(modelId);
+ this.stopRequests.delete(modelId);
+ this.rejectStatus(modelId, new Error(`Model removed: ${this.host.toDisplayName(modelId)}`));
+ }
+
+ // refetch even when the row is unknown: the server removed something our
+ // list does not know about, so the list is stale either way
void this.host.fetchRouterModels();
}
@@ -638,7 +820,6 @@ export class ModelStatusManager {
private setPausedDownload(repoWithTag: string, progress: ModelDownloadProgress | null): void {
this.pausedDownloads.set(repoWithTag, progress);
- this.persistPausedDownloads();
}
// reassign the array: mutating an entry in place would not trigger reactivity
diff --git a/tools/ui/src/lib/stores/settings/index.svelte.ts b/tools/ui/src/lib/stores/settings/index.svelte.ts
index 765f2e302..d9ed790ec 100644
--- a/tools/ui/src/lib/stores/settings/index.svelte.ts
+++ b/tools/ui/src/lib/stores/settings/index.svelte.ts
@@ -201,6 +201,8 @@ class SettingsStore {
// Restore user overrides (derived state — may be stale if server defaults differ)
this.userOverrides = new Set(data.userOverrides ?? []);
+ this.enforceDiscoverCoupling();
+
// Persist to localStorage
this.saveConfig();
@@ -220,6 +222,7 @@ class SettingsStore {
try {
this.loadConfig();
this.migrateLegacyTheme();
+ this.enforceDiscoverCoupling();
// Apply the persisted theme from config on initial load
setMode(this.config[SETTINGS_KEYS.THEME] as ColorMode);
this.isInitialized = true;
@@ -243,6 +246,7 @@ class SettingsStore {
resetConfig() {
this.config = { ...SETTING_CONFIG_DEFAULT };
+ this.enforceDiscoverCoupling();
this.saveConfig();
}
@@ -362,14 +366,6 @@ class SettingsStore {
this.saveConfig();
}
- /**
- *
- *
- * Import / Export
- *
- *
- */
-
/**
* Update multiple configuration settings at once
* @param updates - Object containing the configuration updates
@@ -399,6 +395,14 @@ class SettingsStore {
this.saveConfig();
}
+ /**
+ *
+ *
+ * Import / Export
+ *
+ *
+ */
+
/**
* Update the theme setting.
* @param newTheme - The new theme value
@@ -409,6 +413,20 @@ class SettingsStore {
setMode(newTheme as ColorMode);
}
+ /**
+ * Discover needs the hub metadata reads, so a checked Discover Models keeps
+ * the hub setting on; call wherever stored or reset state may disagree.
+ */
+ private enforceDiscoverCoupling(): void {
+ if (
+ this.config[SETTINGS_KEYS.ENABLE_DISCOVER_MODELS] &&
+ !this.config[SETTINGS_KEYS.USE_HUGGING_FACE_HUB]
+ ) {
+ setConfigValue(this.config, SETTINGS_KEYS.USE_HUGGING_FACE_HUB, true);
+ this.saveConfig();
+ }
+ }
+
/**
*
*
diff --git a/tools/ui/src/lib/stores/ui.svelte.ts b/tools/ui/src/lib/stores/ui.svelte.ts
index 2578a6c17..bd9e40323 100644
--- a/tools/ui/src/lib/stores/ui.svelte.ts
+++ b/tools/ui/src/lib/stores/ui.svelte.ts
@@ -12,6 +12,10 @@ class UiStore {
/** Set when a flow outside the chat wants the composer focused once it can take it. */
composerFocusRequested = $state(false);
+ /** Repo the Discover view selects when it opens, set by a model row. */
+ discoverModelFocus = $state<string | null>(null);
+ discoverModelsOpen = $state(false);
+
/** Whether the desktop sidebar is expanded (open). */
isSidebarExpanded = $state(false);
/** Model the manager reveals when it opens, a qualified id or a raw model name. */
@@ -26,6 +30,12 @@ class UiStore {
this.modelInformation = option;
}
+ /** Open the models dialog on Discover, focused on one repo. */
+ openModelsDiscover(focus?: string): void {
+ this.discoverModelFocus = focus ?? null;
+ this.discoverModelsOpen = true;
+ }
+
/** Open the models manager, optionally focused on one model. */
openModelsManager(focus?: string): void {
this.manageModelFocus = focus ?? null;
diff --git a/tools/ui/src/lib/types/api.d.ts b/tools/ui/src/lib/types/api.d.ts
index 70b99eaa0..853280fb5 100644
--- a/tools/ui/src/lib/types/api.d.ts
+++ b/tools/ui/src/lib/types/api.d.ts
@@ -1,4 +1,5 @@
import type { ChatMessagePromptProgress, ChatRole } from './chat';
+import type { ModelSidecar } from '$lib/constants';
import type {
ContentPartType,
FileTypeAudio,
@@ -102,6 +103,8 @@ export interface ApiModelDataEntry {
architecture?: ApiModelArchitecture;
/** Trained context of the model, read from its GGUF metadata at registration */
context_length?: number;
+ /** Sidecar kinds a cached model or an unfinished download has on disk, as -hf resolves them */
+ sidecars?: ModelSidecar[];
/** Legacy meta field (may be present in older responses) */
meta?: Record<string, unknown> | null;
}
@@ -170,6 +173,8 @@ export interface ApiModelsSseEvent {
*/
export interface ApiModelsDownloadRequest {
model: string;
+ /** Sidecar kinds pulled with the model, picked by the server as -hf picks them */
+ sidecars: ModelSidecar[];
}
/**
diff --git a/tools/ui/src/lib/types/huggingface.d.ts b/tools/ui/src/lib/types/huggingface.d.ts
index 5821097cb..d578741c4 100644
--- a/tools/ui/src/lib/types/huggingface.d.ts
+++ b/tools/ui/src/lib/types/huggingface.d.ts
@@ -229,6 +229,8 @@ export interface HfCatalogEntry {
released: string;
license: string;
featured?: boolean;
+ /** Decision models answer with probabilities, not text; llama-ui cannot use them. */
+ decision?: boolean;
maxMemGb?: number;
sizes: HfCatalogSize[];
}
diff --git a/tools/ui/src/lib/types/index.ts b/tools/ui/src/lib/types/index.ts
index bda7371cb..820165cc9 100644
--- a/tools/ui/src/lib/types/index.ts
+++ b/tools/ui/src/lib/types/index.ts
@@ -109,9 +109,20 @@ export type {
ModelDownloadFileProgress,
ModelDownloadProgress,
ModelLoadProgress,
+ ModelSidecarBadge,
ModalityCapabilities
} from './models';
+// Models discover types
+export type {
+ ModelBitDepthRow,
+ ModelCompanionDraft,
+ ModelDownloadEntryState,
+ ModelQuantChip,
+ ModelQuantOption,
+ ModelSelectableFile
+} from './models-discover';
+
// Settings types
export type {
SettingsConfigValue,
diff --git a/tools/ui/src/lib/types/models-discover.d.ts b/tools/ui/src/lib/types/models-discover.d.ts
new file mode 100644
index 000000000..71a56f54c
--- /dev/null
+++ b/tools/ui/src/lib/types/models-discover.d.ts
@@ -0,0 +1,56 @@
+import type { ModelSidecar } from '$lib/constants';
+import type { ModelDraftSidecar, ModelSelectableFileKind } from '$lib/enums';
+import type { HfModelSibling } from '$lib/types/huggingface';
+
+/**
+ * Option of a quant `<select>` in the download-options command builder; the
+ * picks only compose the serve command and are not bound to the quant chips.
+ */
+export interface ModelQuantOption {
+ /** Quant token, or the file name when the file carries no quant (e.g. BF16). */
+ label: string;
+ path: string;
+}
+
+/** Download state of a single repo entry, injected by the integration layer. */
+export interface ModelDownloadEntryState {
+ /** Server identifier the entry's actions (pause / resume / cancel / retry) target. */
+ repoWithTag: string;
+ isDownloading: boolean;
+ progress: ModelDownloadProgress | null;
+ isDownloaded: boolean;
+ isPaused: boolean;
+ isFailed: boolean;
+}
+
+/** A group of GGUF files sharing one bit-depth bucket. */
+export type ModelBitDepthRow = { bitDepth: number; files: HfModelSibling[] };
+
+/** A selectable GGUF, tagged with its role: main weights, draft, or aux (mmproj). */
+export type ModelSelectableFile = HfModelSibling & { kind: ModelSelectableFileKind };
+
+/**
+ * Draft file joined to a main quant: the server pulls the two together, so a
+ * chip carries its draft as a badge instead of a download of its own.
+ */
+export interface ModelCompanionDraft {
+ /** Draft sidecar kind, e.g. `mtp`. */
+ kind: ModelDraftSidecar;
+ /** True when the listing reports the kind already on disk. */
+ onDisk: boolean;
+ /** Quant of the draft file itself; null when its name carries none. */
+ quant: string | null;
+ path: string;
+ size: number;
+}
+
+/** A main quant chip: its file, its download state and the draft that rides with it. */
+export interface ModelQuantChip extends ModelSelectableFile {
+ /** Draft the chip pulls with the model, badged on the chip; null when none of its kind matches. */
+ companion: ModelCompanionDraft | null;
+ /** Quant of the chip's own file; null when its name carries none. */
+ quant: string | null;
+ /** Sidecar kinds the download posts on top of the model's own; the joined draft. */
+ sidecars: ModelSidecar[];
+ state: ModelDownloadEntryState;
+}
diff --git a/tools/ui/src/lib/types/models.d.ts b/tools/ui/src/lib/types/models.d.ts
index 579ea2327..ee17a0c5c 100644
--- a/tools/ui/src/lib/types/models.d.ts
+++ b/tools/ui/src/lib/types/models.d.ts
@@ -76,6 +76,8 @@ export interface ModelSidecarBadge {
quant: string | null;
/** Repo the sidecar file belongs to, which may differ from the model's. */
repo: string;
+ /** True while the draft is still coming down; a download row reads it as in flight. */
+ pending?: boolean;
}
/** Modality capabilities for file validation. */
diff --git a/tools/ui/src/lib/utils/index.ts b/tools/ui/src/lib/utils/index.ts
index 4d7bca4de..5b46bedef 100644
--- a/tools/ui/src/lib/utils/index.ts
+++ b/tools/ui/src/lib/utils/index.ts
@@ -119,7 +119,14 @@ export {
export { isValidModelName, normalizeModelName, orgOf, repoOf } from './model-names';
// Sidecar token utilities
-export { isAuxSidecar, isDraftSidecar, sidecarFromFileToken, sidecarFromTag } from './sidecars';
+export {
+ directoryOf,
+ isAuxSidecar,
+ isDraftSidecar,
+ isServerSidecarFile,
+ sidecarFromFileToken,
+ sidecarFromTag
+} from './sidecars';
// Portal utilities
export { portalToBody } from './portal-to-body';
diff --git a/tools/ui/src/lib/utils/sidecars.ts b/tools/ui/src/lib/utils/sidecars.ts
index 52532b135..f89ba4d6b 100644
--- a/tools/ui/src/lib/utils/sidecars.ts
+++ b/tools/ui/src/lib/utils/sidecars.ts
@@ -1,4 +1,4 @@
-import { MODEL_ID, type ModelSidecar, SIDECAR_TOKENS } from '$lib/constants';
+import { MODEL_ID, type ModelSidecar, PATH_SEPARATOR, SIDECAR_TOKENS } from '$lib/constants';
import { ModelAuxSidecar, ModelDraftSidecar } from '$lib/enums';
const SIDECAR_TOKEN_SET = new Set<string>(SIDECAR_TOKENS);
@@ -28,3 +28,19 @@ export function isDraftSidecar(sidecar: ModelSidecar): sidecar is ModelDraftSide
export function isAuxSidecar(sidecar: ModelSidecar): sidecar is ModelAuxSidecar {
return AUX_SIDECAR_SET.has(sidecar);
}
+
+/** Directory of a repo-relative path, e.g. `MTP/mtp-x.gguf` -> `MTP`; empty at the repo root. */
+export function directoryOf(path: string): string {
+ const separator = path.lastIndexOf(PATH_SEPARATOR);
+
+ return separator === -1 ? '' : path.slice(0, separator);
+}
+
+/**
+ * True when llama-server's sidecar lookup matches `kind` in this file name: the
+ * case-sensitive `<kind>-` token find_best_sibling scans for. Mirrored here so
+ * the UI never offers a draft the server cannot resolve.
+ */
+export function isServerSidecarFile(path: string, kind: string): boolean {
+ return path.includes(`${kind}-`);
+}
diff --git a/tools/ui/tests/client/markdown-content-sanitize.svelte.test.ts b/tools/ui/tests/client/markdown-content-sanitize.svelte.test.ts
new file mode 100644
index 000000000..1df5a65e3
--- /dev/null
+++ b/tools/ui/tests/client/markdown-content-sanitize.svelte.test.ts
@@ -0,0 +1,160 @@
+// Adversarial checks for MarkdownContent's allowHtml mode: the discover README
+// renders HuggingFace model cards, which are third-party content, so every
+// payload below must come out neutralized - both right after sanitize and
+// after an innerHTML re-parse, which is where mutation XSS would surface.
+import MarkdownContent from '$lib/components/app/content/MarkdownContent/MarkdownContent.svelte';
+import { SAFE_HTML_CONFIG } from '$lib/components/app/content/MarkdownContent/safe-html-config';
+import DOMPurify from 'dompurify';
+import { mount, unmount } from 'svelte';
+import { describe, expect, it } from 'vitest';
+
+const HANDLER_ATTR = /\son[a-z]+\s*=/i;
+const JS_URL = /javascript\s*:/i;
+
+/** Payload execution canary: set by any payload that runs. */
+function xssFired(): boolean {
+ return (window as { __mdXss?: number }).__mdXss !== undefined;
+}
+
+function roundTrip(html: string): string {
+ const el = document.createElement('div');
+
+ el.innerHTML = html;
+ document.body.appendChild(el);
+ const again = el.innerHTML;
+
+ el.remove();
+
+ return again;
+}
+
+const PAYLOADS: Record<string, string> = {
+ 'details ontoggle': '<details open ontoggle="window.__mdXss=1">x</details>',
+ 'dom clobbering': '<p id="content" name="location"><input name="domain"></p>',
+ 'form and input': '<form><input autofocus onfocus="window.__mdXss=1"></form>',
+ 'iframe javascript src': '<iframe src="javascript:alert(1)"></iframe>',
+ 'iframe srcdoc': '<iframe srcdoc="<script>window.__mdXss=1</script>"></iframe>',
+ 'img onerror': '<img src=x onerror="window.__mdXss=1">',
+ 'link case and entity href': '<a href="JaVaScRiPt:alert(1)">x</a>',
+ 'link data:text/html href':
+ '<a href="data:text/html;base64,PHNjcmlwdD5hbGVydCgxKTwvc2NyaXB0Pg==">x</a>',
+ 'link javascript href': '<a href="javascript:alert(1)">x</a>',
+ 'link vbscript href': '<a href="vbscript:msgbox(1)">x</a>',
+ 'math href': '<math href="javascript:alert(1)"></math>',
+ 'mathml annotation-xml integration point':
+ '<math><annotation-xml encoding="text/html"><img src=x onerror="window.__mdXss=1"></annotation-xml></math>',
+ 'mathml mtext style mXSS':
+ '<math><mtext><table><mglyph><style><img src=x onerror="window.__mdXss=1"></style></mglyph></table></mtext></math>',
+ noscript: '<noscript><p title="</noscript><img src=x onerror="window.__mdXss=1>">',
+ script: '<script>window.__mdXss=1</script>',
+ 'srcset javascript': '<img srcset="javascript:alert(1) 1x, x 2x" src=x>',
+ 'style tag': '<style>@import url(javascript:alert(1));</style>',
+ 'svg foreignObject smuggling':
+ '<svg><foreignObject><img src=x onerror="window.__mdXss=1"></foreignObject></svg>',
+ 'svg set-attribute animation': '<svg><set attributeName="onmouseover" to="alert(1)"/></svg>',
+ 'unclosed tag': '<img src="x" onerror="window.__mdXss=1"',
+ 'uppercase tags': '<IMG SRC=x ONERROR="window.__mdXss=1">',
+ 'video source onerror': '<video><source onerror="window.__mdXss=1" src=x></video>',
+ 'xlink href': '<math><mtext xlink:href="javascript:alert(1)"></mtext></math>'
+};
+
+describe('SAFE_HTML_CONFIG adversarial battery', () => {
+ it('neutralizes every payload after sanitize and after innerHTML re-parse', () => {
+ const failures: string[] = [];
+
+ for (const [name, payload] of Object.entries(PAYLOADS)) {
+ const clean = DOMPurify.sanitize(payload, SAFE_HTML_CONFIG) as string;
+ const again = roundTrip(clean);
+
+ if (HANDLER_ATTR.test(clean) || HANDLER_ATTR.test(again)) {
+ failures.push(`${name}: handler attribute survived: ${again}`);
+ }
+
+ if (JS_URL.test(clean) || JS_URL.test(again)) {
+ failures.push(`${name}: javascript: URL survived: ${again}`);
+ }
+ }
+
+ expect(failures).toEqual([]);
+ });
+
+ it('scrubs positioned styles that can draw an overlay, keeps KaTeX styles', () => {
+ const clean = DOMPurify.sanitize(
+ '<div style="position:fixed;inset:0;z-index:9999;background:red">x</div><span style="height:0.83em;vertical-align:-0.2em">y</span>',
+ SAFE_HTML_CONFIG
+ ) as string;
+
+ expect(clean).not.toMatch(/position\s*:\s*fixed/i);
+ expect(clean).toContain('height:0.83em');
+ expect(clean).toContain('background:red');
+ });
+
+ it('keeps benign model-card markup intact', () => {
+ const clean = DOMPurify.sanitize(
+ '<h2 id="title">Title</h2><p>Text with <a href="https://example.com" target="_blank" rel="noopener">a link</a>, <img src="https://example.com/i.png" alt="i" width="100">, <code>code</code>, <table><tr><td colspan="2">cell</td></tr></table>, <math><semantics><annotation encoding="application/x-tex">x^2</annotation></semantics></math></p>',
+ SAFE_HTML_CONFIG
+ ) as string;
+
+ expect(clean).toContain('href="https://example.com"');
+ expect(clean).toContain('<img');
+ expect(clean).toContain('colspan="2"');
+ expect(clean).toContain('<annotation');
+ });
+});
+
+describe('MarkdownContent end to end', () => {
+ it('renders an allowHtml payload README without executing it', async () => {
+ const evil = [
+ '# Evil card',
+ '',
+ '<img src=x onerror="window.__mdXss=1">',
+ '<a href="javascript:alert(1)">x</a>',
+ '<script>window.__mdXss=1</script>',
+ '<iframe srcdoc="<script>window.__mdXss=1</script>"></iframe>',
+ '<math><annotation-xml encoding="text/html"><img src=x onerror="window.__mdXss=1"></annotation-xml></math>',
+ '',
+ 'normal **markdown** continues'
+ ].join('\n');
+ const target = document.createElement('div');
+
+ document.body.appendChild(target);
+ mount(MarkdownContent, { props: { allowHtml: true, content: evil }, target });
+
+ for (let i = 0; i < 100 && !target.textContent?.includes('normal'); i++) {
+ await new Promise((r) => setTimeout(r, 50));
+ }
+ await new Promise((r) => setTimeout(r, 200));
+
+ expect(target.textContent).toContain('normal');
+ expect(xssFired()).toBe(false);
+ expect(target.querySelector('script')).toBeNull();
+ expect(target.querySelector('iframe')).toBeNull();
+
+ target.remove();
+ });
+
+ it('escapes raw HTML to literal text in the default mode', async () => {
+ const target = document.createElement('div');
+
+ document.body.appendChild(target);
+ const component = mount(MarkdownContent, {
+ props: {
+ content: '<script>window.__mdXss=1</script><img src=x onerror="window.__mdXss=1">'
+ },
+ target
+ });
+
+ for (let i = 0; i < 100 && !target.textContent?.includes('onerror'); i++) {
+ await new Promise((r) => setTimeout(r, 50));
+ }
+ await new Promise((r) => setTimeout(r, 200));
+
+ expect(xssFired()).toBe(false);
+ expect(target.querySelector('img[src="x"]')).toBeNull();
+ expect(target.textContent).toContain('onerror');
+
+ target.remove();
+
+ if (component) unmount(component);
+ });
+});
diff --git a/tools/ui/tests/client/models-download-flows.svelte.test.ts b/tools/ui/tests/client/models-download-flows.svelte.test.ts
new file mode 100644
index 000000000..42843c944
--- /dev/null
+++ b/tools/ui/tests/client/models-download-flows.svelte.test.ts
@@ -0,0 +1,155 @@
+// Guards the download flows the chips depend on: one POST per click, the chip
+// reading downloaded once the list reports the model, a resume keeping the
+// sidecars it started with, a cancel dropping the whole download, and a reload
+// restoring the paused state of an unfinished one.
+
+import { ModelAuxSidecar, ModelDraftSidecar, ServerModelStatus, ServerRole } from '$lib/enums';
+import { ModelsService } from '$lib/services/models.service';
+import { modelsStore } from '$lib/stores';
+import { serverStore } from '$lib/stores/server.svelte';
+import type { ApiModelDataEntry, ApiModelsListResponse } from '$lib/types/api';
+import { beforeEach, describe, expect, it, vi } from 'vitest';
+
+const REPO = 'org/model-GGUF';
+const TAG = `${REPO}:Q4_K_M`;
+const DRAFTS = [ModelAuxSidecar.MMPROJ, ModelDraftSidecar.DFLASH];
+
+/** The listing entry a router reports once a tag is downloaded. */
+function entry(id: string, sidecars: string[] = [], status = ServerModelStatus.DOWNLOADED) {
+ return {
+ created: 0,
+ id,
+ in_cache: true,
+ object: 'model',
+ owned_by: 'llamacpp',
+ path: `/models/${id}`,
+ sidecars,
+ status: { value: status }
+ } as unknown as ApiModelDataEntry;
+}
+
+/** The download endpoints the store talks to; the feed is not live in these tests. */
+function stubEndpoints() {
+ const post = vi.spyOn(ModelsService, 'downloadModel').mockResolvedValue({ success: true });
+ const cancel = vi.spyOn(ModelsService, 'cancelDownload').mockResolvedValue({ success: true });
+ const unload = vi.spyOn(ModelsService, 'unload').mockResolvedValue({ success: true });
+
+ vi.spyOn(ModelsService, 'watchModelEvents').mockResolvedValue(undefined);
+ vi.spyOn(ModelsService, 'list').mockResolvedValue({
+ data: [],
+ models: [],
+ object: 'list'
+ } as unknown as ApiModelsListResponse);
+
+ return { cancel, post, unload };
+}
+
+describe('model download flows', () => {
+ beforeEach(() => {
+ vi.restoreAllMocks();
+ serverStore.role = ServerRole.ROUTER;
+ modelsStore.routerModels = [];
+ // the store is a singleton: drop what an earlier case tracked
+ modelsStore.status.unsubscribe();
+ });
+
+ it('posts once for two quick clicks on the same chip', async () => {
+ const { post } = stubEndpoints();
+ const first = modelsStore.status.downloadModel(TAG, [ModelDraftSidecar.DFLASH]);
+ const second = modelsStore.status.downloadModel(TAG, [ModelDraftSidecar.DFLASH]);
+
+ // the chip reads in flight before the server answers, so the second click
+ // finds it busy instead of posting again
+ expect(modelsStore.status.isDownloadInProgress(TAG)).toBe(true);
+
+ await Promise.all([first, second]);
+
+ expect(post).toHaveBeenCalledTimes(1);
+ expect(post).toHaveBeenCalledWith(TAG, DRAFTS);
+ });
+
+ it('reads the chip as downloaded once the list reports the model', async () => {
+ stubEndpoints();
+
+ await modelsStore.status.downloadModel(TAG, [ModelDraftSidecar.DFLASH]);
+
+ expect(modelsStore.status.isModelDownloaded(TAG)).toBe(false);
+
+ // the feed's reload lands as a listing the tag now appears in
+ modelsStore.routerModels = [entry(TAG, DRAFTS)];
+
+ expect(modelsStore.status.isModelDownloaded(TAG)).toBe(true);
+ // the sidecar option of that model reads the draft off the same entry
+ expect(modelsStore.status.isModelDownloaded(`${TAG}-dflash`)).toBe(true);
+ });
+
+ it('does not pause a download whose POST is still on its way', async () => {
+ const { post, unload } = stubEndpoints();
+
+ let release: () => void = () => {};
+
+ post.mockImplementation(
+ () =>
+ new Promise((resolve) => {
+ release = () => resolve({ success: true });
+ })
+ );
+
+ const started = modelsStore.status.downloadModel(TAG, [ModelDraftSidecar.DFLASH]);
+
+ // the chip reads as downloading before the server answers, so a second click
+ // must not stop a model that never started
+ await modelsStore.status.pauseDownload(TAG);
+
+ expect(unload).not.toHaveBeenCalled();
+ expect(modelsStore.status.isDownloadInProgress(TAG)).toBe(true);
+ expect(modelsStore.status.isDownloadPaused(TAG)).toBe(false);
+
+ release();
+ await started;
+ });
+
+ it('resumes with the sidecars the download started with', async () => {
+ const { post } = stubEndpoints();
+
+ await modelsStore.status.downloadModel(TAG, [ModelDraftSidecar.DFLASH]);
+ await modelsStore.status.pauseDownload(TAG);
+
+ expect(modelsStore.status.isDownloadPaused(TAG)).toBe(true);
+
+ // the resume a chip sends names no kinds of its own
+ await modelsStore.status.downloadModel(TAG);
+
+ expect(post).toHaveBeenLastCalledWith(TAG, DRAFTS);
+ expect(modelsStore.status.isDownloadPaused(TAG)).toBe(false);
+ });
+
+ it('cancels the model and its sidecars as one download', async () => {
+ const { cancel } = stubEndpoints();
+
+ await modelsStore.status.downloadModel(TAG, [ModelDraftSidecar.DFLASH]);
+ await modelsStore.status.cancelDownload(TAG);
+
+ // no sidecar of its own is asked for: the model goes as one unit
+ expect(cancel).toHaveBeenCalledWith(TAG, null);
+ expect(modelsStore.status.isDownloadInProgress(TAG)).toBe(false);
+ expect(modelsStore.status.getDownloadEntries()).toEqual([]);
+ });
+
+ it('restores the paused state of an unfinished download after a reload', async () => {
+ const { post } = stubEndpoints();
+
+ modelsStore.routerModels = [entry(TAG, DRAFTS, ServerModelStatus.INCOMPLETE)];
+ modelsStore.status.syncPausedDownloads();
+
+ expect(modelsStore.status.isDownloadPaused(TAG)).toBe(true);
+ expect(modelsStore.status.getDownloadEntries()).toEqual([
+ { isPaused: true, progress: null, repoWithTag: TAG }
+ ]);
+
+ // and its resume keeps the kinds the incomplete listing reports
+ await modelsStore.status.downloadModel(TAG);
+
+ expect(post).toHaveBeenCalledWith(TAG, DRAFTS);
+ });
+});
diff --git a/tools/ui/tests/client/models-manager-utils.svelte.test.ts b/tools/ui/tests/client/models-manager-utils.svelte.test.ts
index 42dd95ce5..f441722ef 100644
--- a/tools/ui/tests/client/models-manager-utils.svelte.test.ts
+++ b/tools/ui/tests/client/models-manager-utils.svelte.test.ts
@@ -9,7 +9,7 @@ import {
splitHiddenQuants
} from '$lib/components/app/models/ModelsManager/utils';
import { LOCAL_BACKEND_ID } from '$lib/constants';
-import { ModelGroupKind, ServerModelStatus } from '$lib/enums';
+import { ModelDraftSidecar, ModelGroupKind, ServerModelStatus } from '$lib/enums';
import { modelsStore } from '$lib/stores/models/index.svelte';
import type { ApiModelDataEntry, ModelOption } from '$lib/types';
import { beforeEach, describe, expect, it } from 'vitest';
@@ -84,23 +84,43 @@ describe('downloadGroups', () => {
const listed = option('org/Qwen3-8B:Q4_K_M');
const groups = downloadGroups(
[{ isPaused: false, progress: null, repoWithTag: 'org/Qwen3-8B:Q4_K_M' }],
- [listed]
+ [listed],
+ () => []
);
expect(groups[0].key).toBe('download::org/Qwen3-8B:Q4_K_M');
- expect(groups[0].base).toBe(listed);
+ expect(groups[0].base).toMatchObject(listed);
});
it('stands the download in for a row of its own when nothing is listed', () => {
const groups = downloadGroups(
[{ isPaused: true, progress: null, repoWithTag: 'org/Qwen3-8B:Q4_K_M' }],
- []
+ [],
+ (repo) =>
+ repo === 'org/Qwen3-8B' ? [{ kind: ModelDraftSidecar.MTP, quant: 'Q8_0', repo }] : []
);
+ // the sub-lines carry the sidecars, so the inline badges stay empty
expect(groups[0].base).toMatchObject({
backendId: LOCAL_BACKEND_ID,
+ draftSidecars: [],
model: 'org/Qwen3-8B:Q4_K_M'
});
+ expect(groups[0].sidecarBadges).toEqual([
+ { kind: ModelDraftSidecar.MTP, quant: 'Q8_0', repo: 'org/Qwen3-8B' }
+ ]);
+ });
+
+ it('keeps one row per tracked download, sidecars included', () => {
+ const main = { isPaused: false, progress: null, repoWithTag: 'org/Qwen3-8B:Q4_K_M' };
+ const other = { isPaused: true, progress: null, repoWithTag: 'org/Qwen3-8B:Q8_0' };
+ const groups = downloadGroups([main, other], [], () => []);
+
+ expect(groups).toHaveLength(2);
+ expect(groups.map((group) => group.key)).toEqual([
+ 'download::org/Qwen3-8B:Q4_K_M',
+ 'download::org/Qwen3-8B:Q8_0'
+ ]);
});
});
diff --git a/tools/ui/tests/client/models-sidecar-badges.svelte.test.ts b/tools/ui/tests/client/models-sidecar-badges.svelte.test.ts
new file mode 100644
index 000000000..a5754abc1
--- /dev/null
+++ b/tools/ui/tests/client/models-sidecar-badges.svelte.test.ts
@@ -0,0 +1,99 @@
+// Guards the draft badges a repo row shows: a loaded model records its draft in
+// the `--model-draft` args, which know the file's quant, and the listing reports
+// the kinds of every cached model, which do not. The row shows each kind once.
+
+import { ModelAuxSidecar, ModelDraftSidecar, ServerModelStatus } from '$lib/enums';
+import { modelsStore } from '$lib/stores';
+import type { ApiModelDataEntry } from '$lib/types/api';
+import { beforeEach, describe, expect, it } from 'vitest';
+
+const REPO = 'ggml-org/Qwen3.8-Flash-Next-GGUF';
+
+/** A router entry; `args` stand in for the args of a loaded instance. */
+function entry(
+ id: string,
+ sidecars: string[] = [],
+ args: string[] = [],
+ status = ServerModelStatus.UNLOADED
+) {
+ return {
+ created: 0,
+ id,
+ in_cache: true,
+ object: 'model',
+ owned_by: 'llamacpp',
+ path: `/models/${id}`,
+ sidecars,
+ status: { args, value: status }
+ } as unknown as ApiModelDataEntry;
+}
+
+describe('draft sidecar badges', () => {
+ beforeEach(() => {
+ modelsStore.routerModels = [];
+ });
+
+ it('shows one badge for a draft the args and the listing both report', () => {
+ modelsStore.routerModels = [
+ entry(
+ `${REPO}:IQ4_NL`,
+ [],
+ [
+ '--model-draft',
+ `/cache/models--ggml-org--Qwen3.8-Flash-Next-GGUF/snapshots/${'a'.repeat(40)}/mtp-Qwen3.8-Flash-Next-Q8_0.gguf`
+ ]
+ ),
+ entry(`${REPO}:Q8_0`, [ModelAuxSidecar.MMPROJ, ModelDraftSidecar.MTP])
+ ];
+
+ // the args badge wins: it knows the quant the listing does not
+ expect(modelsStore.status.getDraftSidecars(REPO)).toEqual([
+ { kind: ModelDraftSidecar.MTP, quant: 'Q8_0', repo: REPO }
+ ]);
+ });
+
+ it('shows one badge for a kind two cached quants report', () => {
+ modelsStore.routerModels = [
+ entry(`${REPO}:IQ4_NL`, [ModelDraftSidecar.MTP]),
+ entry(`${REPO}:Q8_0`, [ModelDraftSidecar.MTP])
+ ];
+
+ expect(modelsStore.status.getDraftSidecars(REPO)).toEqual([
+ { kind: ModelDraftSidecar.MTP, quant: null, repo: REPO }
+ ]);
+ });
+
+ it('marks the drafts of a download on its way as pending', () => {
+ modelsStore.routerModels = [
+ entry(
+ `${REPO}:Q4_0`,
+ [ModelAuxSidecar.MMPROJ, ModelDraftSidecar.MTP],
+ [],
+ ServerModelStatus.DOWNLOADING
+ )
+ ];
+
+ // the kinds are not on disk yet, so the row reads them as in flight
+ expect(modelsStore.status.getDraftSidecars(REPO)).toEqual([
+ { kind: ModelDraftSidecar.MTP, pending: true, quant: null, repo: REPO }
+ ]);
+ });
+
+ it('keeps the on-disk badge of a kind a download also pulls', () => {
+ modelsStore.routerModels = [
+ entry(`${REPO}:IQ4_NL`, [ModelDraftSidecar.MTP], [], ServerModelStatus.INCOMPLETE),
+ entry(
+ `${REPO}:Q8_0`,
+ [],
+ [
+ '--model-draft',
+ `/cache/models--ggml-org--Qwen3.8-Flash-Next-GGUF/snapshots/${'a'.repeat(40)}/mtp-Qwen3.8-Flash-Next-Q8_0.gguf`
+ ]
+ )
+ ];
+
+ expect(modelsStore.status.getDraftSidecars(REPO)).toEqual([
+ { kind: ModelDraftSidecar.MTP, quant: 'Q8_0', repo: REPO }
+ ]);
+ });
+});
diff --git a/tools/ui/tests/unit/hf-helpers.test.ts b/tools/ui/tests/unit/hf-helpers.test.ts
index d73a8de22..96ec9afd0 100644
--- a/tools/ui/tests/unit/hf-helpers.test.ts
+++ b/tools/ui/tests/unit/hf-helpers.test.ts
@@ -113,6 +113,43 @@ describe('parseParamCount', () => {
});
});
+describe('isDecisionModel', () => {
+ it('flags the decision-model tag', () => {
+ expect(
+ HuggingFaceService.isDecisionModel({
+ pipeline_tag: 'text-classification',
+ tags: ['gguf', 'decision-model', 'text-classification']
+ })
+ ).toBe(true);
+ });
+
+ it('flags a text-classification pipeline even without the tag', () => {
+ expect(
+ HuggingFaceService.isDecisionModel({ pipeline_tag: 'text-classification', tags: ['gguf'] })
+ ).toBe(true);
+ });
+
+ it('passes regular LLM tasks through', () => {
+ expect(
+ HuggingFaceService.isDecisionModel({
+ pipeline_tag: 'text-generation',
+ tags: ['gguf', 'text-generation']
+ })
+ ).toBe(false);
+
+ expect(
+ HuggingFaceService.isDecisionModel({
+ pipeline_tag: 'image-text-to-text',
+ tags: ['gguf', 'image-text-to-text']
+ })
+ ).toBe(false);
+ });
+
+ it('passes repos without task metadata through', () => {
+ expect(HuggingFaceService.isDecisionModel({ pipeline_tag: null, tags: [] })).toBe(false);
+ });
+});
+
describe('parseSizeBytes', () => {
it('parses single-letter catalog size suffixes', () => {
expect(parseSizeBytes('177g')).toBe(177e9);
diff --git a/tools/ui/tests/unit/model-id-parser.test.ts b/tools/ui/tests/unit/model-id-parser.test.ts
index 3e2a1c5c6..dc03e55b5 100644
--- a/tools/ui/tests/unit/model-id-parser.test.ts
+++ b/tools/ui/tests/unit/model-id-parser.test.ts
@@ -1,3 +1,4 @@
+import type { ModelSidecar } from '$lib/constants';
import { ModelAuxSidecar, ModelDraftSidecar } from '$lib/enums';
import { ModelsService } from '$lib/services/models.service';
import { describe, expect, it } from 'vitest';
@@ -100,6 +101,32 @@ describe('parseModelId', () => {
});
});
+ it('splits a download tag into quant and sidecar', () => {
+ expect(parseModelId('ggml-org/Qwen3.8-27B-GGUF:Q8_0-dflash')).toMatchObject({
+ orgName: 'ggml-org',
+ params: '27B',
+ quantization: 'Q8_0',
+ sidecar: 'dflash'
+ });
+
+ expect(parseModelId('ggml-org/Qwen3.8-27B-GGUF:Q4_K_M-mtp')).toMatchObject({
+ quantization: 'Q4_K_M',
+ sidecar: 'mtp'
+ });
+
+ // a bare sidecar tag is the sidecar alone, with no quant
+ expect(parseModelId('ggml-org/Qwen3.8-27B-GGUF:mmproj')).toMatchObject({
+ quantization: null,
+ sidecar: 'mmproj'
+ });
+
+ // a plain quant tag stays a quant
+ expect(parseModelId('unsloth/DeepSeek-V4-Flash-0731-GGUF:UD-Q2_K_XL')).toMatchObject({
+ quantization: 'UD-Q2_K_XL',
+ sidecar: null
+ });
+ });
+
it('strips trailing container format segments from model names', () => {
expect(parseModelId('unsloth/DeepSeek-V4-Flash-0731-GGUF:Q2_K_XL')).toStrictEqual({
activatedParams: null,
@@ -362,3 +389,35 @@ describe('parseModelId', () => {
});
});
});
+
+describe('draftSidecarsByRepo', () => {
+ const entry = (id: string, sidecars: ModelSidecar[], status = 'unloaded') => ({
+ id,
+ sidecars,
+ status: { value: status }
+ });
+
+ it('keeps one badge per kind, at the repo that has it on disk', () => {
+ const byRepo = ModelsService.draftSidecarsByRepo([
+ entry('ggml-org/model:Q4_0', [ModelDraftSidecar.MTP]),
+ entry('ggml-org/model:Q8_0', [ModelDraftSidecar.MTP])
+ ]);
+
+ // the listing reports kinds, never the draft's file: its quant stays unknown
+ expect(byRepo.get('ggml-org/model')).toEqual([
+ { kind: ModelDraftSidecar.MTP, quant: null, repo: 'ggml-org/model' }
+ ]);
+ });
+
+ it('drops duplicates, aux sidecars and unfinished downloads', () => {
+ const byRepo = ModelsService.draftSidecarsByRepo([
+ entry('ggml-org/model:Q4_0', [ModelDraftSidecar.MTP, ModelAuxSidecar.MMPROJ]),
+ entry('ggml-org/model:Q4_0', [ModelDraftSidecar.MTP]),
+ entry('ggml-org/model:Q8_0', [ModelDraftSidecar.MTP], 'incomplete')
+ ]);
+
+ expect(byRepo.get('ggml-org/model')).toEqual([
+ { kind: ModelDraftSidecar.MTP, quant: null, repo: 'ggml-org/model' }
+ ]);
+ });
+});
diff --git a/tools/ui/tests/unit/model-sidecar-grammar.test.ts b/tools/ui/tests/unit/model-sidecar-grammar.test.ts
index 1ad59171a..aa1148ee5 100644
--- a/tools/ui/tests/unit/model-sidecar-grammar.test.ts
+++ b/tools/ui/tests/unit/model-sidecar-grammar.test.ts
@@ -4,7 +4,7 @@ import { ModelsService } from '$lib/services/models.service';
import { sidecarFromTag } from '$lib/utils';
import { describe, expect, it } from 'vitest';
-const { buildDownloadTag, isSidecarEntry } = ModelsService;
+const { buildDownloadTag } = ModelsService;
const { extractQuantMeta } = HuggingFaceService;
// the sidecar filename grammar mirrors the server (common/download.cpp):
@@ -115,18 +115,6 @@ describe('buildDownloadTag', () => {
});
});
-describe('isSidecarEntry', () => {
- it('detects sidecar entries by their tag', () => {
- expect(isSidecarEntry('org/repo:Q4_0-mtp')).toBe(true);
- expect(isSidecarEntry('org/repo:mmproj')).toBe(true);
- });
-
- it('leaves plain model entries loadable', () => {
- expect(isSidecarEntry('org/repo:Q4_0')).toBe(false);
- expect(isSidecarEntry('org/repo')).toBe(false);
- });
-});
-
describe('sidecarFromTag', () => {
it('reads the token after the last dash', () => {
expect(sidecarFromTag('Q4_0-mtp')).toBe(ModelDraftSidecar.MTP);