Normalize the search term

[joelit]

Fixes #1184

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

[codecov]

Codecov Report

Merging #1239 (5bfe646) into master (30ce5d0) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##             master    #1239   +/-   ##
=========================================
  Coverage     69.07%   69.08%           
  Complexity     1638     1638           
=========================================
  Files            32       32           
  Lines          4016     4017    +1     
=========================================
+ Hits           2774     2775    +1     
  Misses         1242     1242           

Impacted Files	Coverage Δ
model/ConceptSearchParameters.php	84.44% <100.00%> (+0.17%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 30ce5d0...5bfe646. Read the comment docs.

[osma]

I tested this and it works as it should. The only issue is that highlighting (with bold text) the matching parts in the autocomplete box is not working when the input string is decomposed (composed on the left, decomposed on the right):

I tried to fix this by adding a .normalize('NFC') call here:

Skosmos/resource/js/docready.js

Line 804 in 30ce5d0

searchString = $(this).val();

...but it didn't help; I think it would require changes within typeahead.js as it seems to read the search string directly from the text field, so there is no easy opportunity to normalize it. Anyway this is no big deal; there are other similar problems with missing highlights (e.g. in case of accent folding) and these should be pretty rare cases anyway.

I think it's reasonable to place the normalization call in ConceptSearchParameters.getSearchTerm(), as that method also performs other types of search term normalization such as stripping whitespace.

[kinow]

I think it would require changes within typeahead.js

I think #1182 is also updating typeahead. I can try to test it with the latest version and comment if we have a follow-up issue for that @osma 👍 (I will see if I have some text with surrogate characters or just manually edit a dataset and upload to my test fuseki)

[osma]

will see if I have some text with surrogate characters or just manually edit a dataset and upload to my test fuseki

You don't need very special data for this, just text with non-ASCII characters (e.g. åäöéñ) that are stored as composed Unicode characters (NFC), which is the usual case for RDF data.

The problem was when the user enters a search string which contains decomposed (NFD) characters (typically copied and pasted from some other system - in our case an Aleph ILS). This comment in the original issue shows how to trigger it with Linux command line tools and the KANTO/FINAF data set in Finto.

[joelit]

Thanks for spotting this defect, and thanks for the valuable input. :) I'll make a new issue regarding the autocomplete highlighting bug, and call this one an incremental improvement on the situation (and I'll link this discussion there).