Fix #5021: add docSearch to coffeescript doc

[s-pace]

Adding the DocSearch experience. We do provide analytics on the search, feel free to ping.

[jashkenas]

Looks like some unrelated changes to the docs snuck into the commit as well?

[s-pace]

👋 @jashkenas

Only the update of package.json

The overflow need to be changed in order to enable the dropdown to be fully seen.

[jashkenas]

I'm wondering about the "this list may be incomplete", and "when compiling folders or globs" bits, as seen in the diff...

[s-pace]

Happy to explain every changes.
It is working fine on my side

[s-pace]

ah ok I see your points.

It only a new line introduced. I can change it if needed

[GeoffreyBooth]

@jashkenas The text changes were introduced by an earlier PR, that modified the source .md files but didn’t update the output HTML files. The plan was to update the output with the next release.

But some of those changes need to change, because of further discussions about Node’s modules support. In particular I don’t want to mention .mjs if it’s going away or becoming discouraged, as now seems possible. I’ll make some edits on this PR before merging it in.

[GeoffreyBooth]

A few things:

• I can’t seem to scroll within the sidebar anymore.
• I’d like the search results pane to match the general styling of our site. I added the body font to it, but the colors need to change too (browns and greens, not blues). I can work on this more soon when I have more time.
• Is it possible for Algolia to honor code blocks/Markdown? Like when searching for “modules”, you see results that mention “import and export statements”. It would be nice if “import” was rendered as import, like a code block.

We can’t merge this in until the next release, as the docs now mention the exponentiation operator and merged-in object spread, which haven’t been released yet (and merging this in will update the docs immediately). But we can get this PR to be merge-ready, and I’ll merge it in along with cutting that release.

[s-pace]

👋 @GeoffreyBooth

Thank you for the commits

I can’t seem to scroll within the sidebar anymore

This is solved thanks to 2647023

I’d like the search results pane to match the general styling of our site. I added the body font to it, but the colors need to change too (browns and greens, not blues). I can work on this more soon when I have more time.

You can override the colours. Would you like to change the blue into your theme's colour? Feel free to ping me and explaining what you would like to achieve.

Is it possible for Algolia to honor code blocks/Markdown? Like when searching for “modules”, you see results that mention “import and export statements”. It would be nice if “import” was rendered as import, like a code block.

It could be possible but it is not possible to scrap code since it introduce a lot of noise. Furthermore the meanings of words isn't always the same as plain text.

We can’t merge this in until the next release, as the docs now mention the exponentiation operator and merged-in object spread, which haven’t been released yet (and merging this in will update the docs immediately). But we can get this PR to be merge-ready, and I’ll merge it in along with cutting that release.

We can integrate it to the current version if you want. Just point me the right commit.

Happy to help

[s-pace]

Happy to help in order to merge it :)

[GeoffreyBooth]

It could be possible but it is not possible to scrap code since it introduce a lot of noise. Furthermore the meanings of words isn’t always the same as plain text.

I’m not sure what you mean by this. I don’t mean that code blocks should be indexed; just that inline code, like this, that’s already being indexed should appear in the search results styled as inline code. So for example, in this:

The words “import” and “export” should be rendered in monospace with a light solid background, similar to how GitHub renders inline code blocks between backticks: import and export. That’s the way they appear in the section that’s being excerpted:

It’s just the HTML <code> tag.

[s-pace]

👋

I meant that we do not scrap and extract the cotent of <code/> elements since it isn't include in the text selector. Indexing code introduce a lot of noise and can be counterproductive. We only scrap element wrapped in matching text selectors

Let us know if you still want us to find a way to have this kind of rendering.

[GeoffreyBooth]

Okay, I’ve styled this, including making it look good (I hope) on both desktop and mobile:
https://rawgit.com/s-pace/coffeescript/feat/add_search/docs/v2/
@jashkenas and others, please feel free to give me notes. Also check it out on mobile. I still need to do proper QA, but I think the design should be good hopefully.

@s-pace, one thing that doesn’t look so good is that we have a lot of repetition in search categories/headings/titles:

I figured out the CSS to remove the left column (the muted text) but then it looks even weirder, as each category matches each heading most of the time. Is there a good way to deal with this?

Also I see that in https://github.com/algolia/docsearch-configs/blob/master/configs/coffeescript.json, the base URL is http://coffeescript.org/v2/. Our canonical URLs don’t use the /v2; is this necessary in your search configuration to avoid indexing the v1 docs? I already added some code to rewrite the URLs as they come back from your API.

[s-pace]

👋 @GeoffreyBooth

Thank you fo the details

Regarding the duplicates, it is mainly coming from your structure.

Feel free to PR your config, we can index both versions and use variable in order to filter over it.

cheers

[GeoffreyBooth]

@s-pace I’m not sure I follow. We only want the current docs indexed, but I want the canonical URLs presented to users (so no /v2/).

When you say the duplicates are coming from our structure, how is that? In the example above, “Functions” is under “Language Reference”. See the tree in the sidebar. We use h1, h2 etc. tags as appropriate. How do I get Algolia to categorize appropriately?

[s-pace]

@GeoffreyBooth

My bad, I got it. Check the changes, it is live.

LMK WDYT

[GeoffreyBooth]

That’s better, though the changelog entries should be under “Changelog”:

Do I have something wrong in my HTML structure?

[s-pace]

The changelog part wasn't consistent but I have fixed it 👇

LMK WDYT

[GeoffreyBooth]

That looks better, thanks:

Though wouldn’t it be better if I just changed the HTML structure so that the special casing for the changelog is unnecessary? Since I can’t trigger a reindex it’s hard to know what it should be.

I guess I should remove the left column, since we only have two levels of headings and not three, so there’s duplication in the two columns of suggestions. I can do that via CSS unless there’s a better way.

[s-pace]

Since there isn't enough depth in the level we can't change it so if you want you can use the CSS.

As for the changes you have made. I am not such they are live because I have the same outcome with the older version. Let's continue with the lates one.

Let me know if you need anything else.

[jashkenas]

@GeoffreyBooth — feel free to make the final decision here, but I think I should go on the record again expressing my mild-to-moderate opposition to this search upgrade.

With this change, every keystroke someone types into CoffeeScript's search box is sent to Algolia's search servers ... an entirely unnecessary invasion of privacy, given that all of the content is present on the page itself.

If we were just using a local, JS, opensource library, or even Algolia's offline version, I'd feel a lot better about this.

[GeoffreyBooth]

I didn’t know there was an offline version. I would be fine with that. What I like about a search box in general (over Cmd/Ctrl-F) is some level of relevancy intelligence, so that a search for “class” will show suggestions weighted by things like a heading of “Classes” rather than just always jumping to the first appearance of the word “class” on the page. If there’s a fully client-side solution that can achieve that, that would be worth exploring.

I don’t think anyone who types into a search input expects their input to not be sent to a server. If they wanted that level of privacy, they would use Cmd/Ctrl-F.

[s-pace]

Hi @jashkenas

Thank you for sharing your concern. Privacy is something we also take very seriously at Algolia. While searches are gonna be proceeded by our servers, we of course don't track end users with any cookie and/or retargeting strategy. You can read more about our privacy policy.

Security is key for a SAAS company like us and we heavily invest in it. You can check our achievements in these regards.

[jashkenas]

I'm a bit sensitive to this issue because of current events. The privacy policy:

• May change at any time.
• States that Algolia collects your search queries, IP address, browser and so on.
• States that Algolia may then associate the information they've gathered with other personal information they've purchased from third parties.
• Says that they may then sell the gathered personal information in the event of a merger or acquisition.
• Also says that they can sell or "disclose" the aggregated personal information, in a non-individually-identifiable way, at any time, for any reason.

That's all well and good, normally. But in this case, because there's no technical reason why we need to be sharing searches with remote servers — all of the content is present on the page (intentionally!) — it makes me uncomfortable.

[GeoffreyBooth]

@jashkenas See https://medium.com/dev-channel/how-to-add-full-text-search-to-your-website-4e9c80ce2bf4

It looks like lunr.js or elasticlunr.js can do full text search entirely client-side. We would generate the search index ahead of time when we generate the docs.

[zdenko]

Also

• Fuse.js
• search-index
• Full Text Search Light
• Norch
• Searchico

[GeoffreyBooth]

@zdenko is there one you’ve used that you recommend?

[zdenko]

@GeoffreyBooth I've used Fuse once or twice before. And, "search-index" looks like a good candidate too.
I can put something together this weekend.

[GeoffreyBooth]

@zdenko I’ve already placed and styled the search input on this branch, so you probably want to start from this branch. The search input is separate from Algolia/docSearch, so it would be easy to preserve while removing docSearch.

[GeoffreyBooth]

P.S. I like the idea of having the search index generated ahead of time during cake doc:site, so that the JS on our docs page isn’t generating that index in the background every time the site loads. Whatever solution you choose should ideally have that feature.

[zdenko]

Yes, search index should be definitely generated during doc:site task.

[s-pace]

Thanks for explaining all your concerns and we do understand them.

We can close this PR as coffeescript documentation move forward with a pure frontend solution.

Thanks again for considering the PR! Happy to help if needed