fix(ui-react): search datasets by variable name against mint_standard_variables (#96) - #100
Merged
Merged
Conversation
…_variables The "Variable names" mode of /datasets/search did not search variable names. The typed term went into CKAN's free-text `q`, which Solr answers from title, description and tags — it does not index `mint_standard_variables`, and it tokenises variable names on `_` and `~`. Against TACC this returned 48 datasets for `groundwater` where 12 carry it, and 0 for `corpus_nlp` where 17 do. Same root cause as #94, on a different call site. The thread wizard needs an exact variable name; this box takes a substring a person typed, so it gets its own predicate rather than a copy. - `packagesMatchingVariableSubstring` matches the annotation case-insensitively and narrows each kept package to the resources that carry the term, as the exact-match path already does. - `searchDatasets` reads the whole catalog via `searchAllPackages`. Client-side matching needs it, and it also removes a silent cap: the old call took CKAN's 100-row default while TACC holds 215 packages, so an empty term listed 100 of them and said nothing. - `buildSearchQuery` loses its `variables` branch. It only ever produced a query Solr cannot answer, and leaving it invites the same bug back. - `Dataset.variables` is read off the resources instead of echoing the search term back. Verified live against ckan.tacc.utexas.edu: groundwater 48 -> 12, corpus_nlp 0 -> 17, empty term 100 -> 215, dataset-name search unchanged. Closes #96
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #96.
The defect
/datasets/searchhas a Variable names mode. It did not search variable names.The typed term went into CKAN's free-text
q. Solr answersqfrom title, descriptionand tags; it does not index
mint_standard_variables, and it tokenises variable names on_and~. Measured against TACC (215 packages, 33 annotated):groundwatercorpus_nlpSame root cause as #94, on a different call site. #94 fixed the thread wizard
(
findDatasets) and deliberately leftsearchDatasetsalone.The fix is not a copy of #94. The thread wizard matches an exact variable name; this
box takes a substring a person typed, so it gets its own predicate.
A second, independent fault
searchDatasetscalledsearchPackages, whose default isrows: 100. TACC holds 215packages, and the page treats an empty term as "list every dataset" — so it listed 100 of
215 and said nothing. Client-side matching needs the whole catalog anyway, so both modes
now go through
searchAllPackages(added by #94, which pages properly).Changes
packagesMatchingVariableSubstring/resourceMatchesVariableSubstring— case-insensitivesubstring match against the annotation, spanning the
_Solr would have split on. Narrowseach kept package to the resources that carry the term, as the exact-match path already
does; both now share one
narrowToMatchingResourceshelper.buildSearchQueryloses itsvariablesbranch. It only ever produced a query Solr cannotanswer. Leaving it in invites the same bug back a third time.
DatasetQueryParameters.variablesbecomesvariableSubstring— a bare term. The old*wildcard*convention was a Lit-ism that never meant anything to CKAN.Dataset.variablesis read off the resources rather than echoing the search term back.Testing
data-catalog-api.test.tscovering both modes, paging past the 100-row default, andthat the variable term is never sent to CKAN as
q.packagesMatchingVariableSubstringcases inckan.test.ts.Verified live against
ckan.tacc.utexas.edufromlocalhost, in the browser: the fourcounts above, plus dataset-name search unchanged (
carrizo→ 8 genuine matches).Note on the legacy UI
Lit's own
/datasets/searchis dead at TACC and this does not regress it. That pagedispatches
queryGeneralDatasets, which POSTs todata_catalog_api + "/find_datasets"(
ui/src/screens/datasets/actions.ts:377) — the legacy MINT Data Catalog route. It nevergoes through
DataCatalogAdapter, sodata_catalog_type = "CKAN"does not redirect it.At TACC that URL answers 405, for both modes.
Scope
Client-only. No change at TACC, no Hasura metadata change.
/datasets/browseand/regions/:id/datasetsare deliberately untouched — separate call sites, andext_bboxis already owned by #97.
🤖 Generated with Claude Code