[CT-157] [Bug] Search in generated docs is not filtering as expected

[crystalro0]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

Customer is using Databricks in dbt Cloud. When docs are generated during their production run, user is seeing no change in the search results when they attempt to filter by their search query.

Secondary issue is that the displayed model size is incorrect. (Ex. model is showing as 23GB in docs, while it's ~8MB in Databricks)

Expected Behavior

Results should be filtered by an inputed search term or query. Also if no results, it should not be showing all available items.

Steps To Reproduce

1. Log into customer account and open Documentation.
2. Attempt to search (ex. fct_credit_test).
3. All results are displayed.

Relevant log output

No response

Environment

- dbt: 1.0

What database are you using dbt with?

other (mention it in "Additional Context")

Additional Context

Databricks

[jtcohen6]

Thanks @crystalro0! I was able to take a peek in. When I try searching, I see these errors start appearing in the console:

That error would be more helpful if I had the source maps (which I don't... because my local npm situation is troubled currently):

(index):73 TypeError: Cannot read properties of null (reading 'toLowerCase')
    at (index):67
    at (index):67
    at st ((index):67)
    at Function.sa ((index):67)
    at Object.n.search ((index):67)
    at (index):93
    at c.$digest ((index):73)
    at c.$apply ((index):73)
    at Object.$$debounceViewValueCommit ((index):73)
    at Object.$setViewValue ((index):73)
(anonymous) @ (index):73
(anonymous) @ (index):73
$digest @ (index):73
$apply @ (index):73
$$debounceViewValueCommit @ (index):73
$setViewValue @ (index):73
c @ (index):73
It @ (index):73
n @ (index):73

It looks like there are a few different spots, related to site search, where we call toLowerCase:

dbt-docs/src/app/components/search/search.js

Lines 106 to 114 in 13bc6bd

	scope.columnFilter = function(columns) {
	var matches = [];
	for (var column in columns) {
	if (column.toLowerCase().indexOf(scope.query.toLowerCase()) != -1) {
	matches.push(column);
	}
	}
	return matches;
	}

dbt-docs/src/app/main/index.js

Lines 162 to 202 in 13bc6bd

	_.each(results, function(result){
	result.overallWeight = 0;
	_.each(Object.keys(criteriaArr), function(criteria){
	if(result.model[criteria] != undefined){
	let count = 0;
	let body = result.model[criteria];
	let query = ($scope.search.query).toLowerCase();
	if(criteria === "columns"){
	_.each(body, function(column){
	let columnName = column.name.toLowerCase();
	let index = 0;
	while(index != -1){
	index = columnName.indexOf(query, index);
	if (index != -1) {
	count++; index++;
	}
	}
	});
	}
	else if(criteria === "tags"){
	_.each(body, function(tag){
	let tagName = tag.toLowerCase();
	let index = 0;
	while(index != -1){
	index = tagName.indexOf(query, index);
	if (index != -1) {
	count++; index++;
	}
	}
	});
	}
	else{
	body = body.toLowerCase();
	let index = 0;
	while(index != -1){
	index = body.indexOf(query, index);
	if(index != -1){
	count++; index++;
	}
	}
	}

This will definitely be easier to debug for someone who can build locally with source maps :) In the meantime, I suspected the issue might be due to Spark/Databricks-specific behavior, whereby database is always null (#104) — but I built docs with a different dbt-spark project locally, and search worked just fine.

I'll transfer this to dbt-docs, since that's where (I imagine) the fix will need to happen.

Secondary issue is that the displayed model size is incorrect. (Ex. model is showing as 23GB in docs, while it's ~8MB in Databricks)

That's a pretty substantial difference! I can open a separate issue in dbt-spark as appropriate, but it would be helpful to get a few more details:

• Is this a Delta table? If not, what's the file type?
• When did the user last run analyze table?
• If they run describe table extended 'name_of_this_table', or show table extended like 'name_of_this_table', what do they see in the results? (This is what dbt is doing behind the scenes, and parsing the number of bytes from the text returned

Update: opened dbt-labs/dbt-spark#284