outside filters with brushes

[timelyportfolio]

Summary

Often in integrated applications with parallel coordinates, I need the ability to filter the parallel coordinates from outside with other controls, such as HTML inputs or supplementary visualizations. Here is a very minimal example demonstrating the concept with a modified version of the original parallel-coordinates. Would you be interested in a pull request targeting this functionality?

Proof of Concept

As I mentioned, for the previous example, I modified the original parallel-coordinates to add outsideFilters to the API as shown in timelyportfolio/parallel-coordinates@40fa0dc. This poc only targeted categorical/discrete filtering, but ideally I envision robust filtering that would cover all data types.

Thoughts

In all the brush modes, we have something like this before determining what is selected.

...
  // Resolves broken examples for now. They expect to get the full dataset back from empty brushes
  if (actives.length === 0) return config.data;
...
  return config.data.filter(d => {
    switch (brushGroup.predicate) {
      case 'AND':
        return actives.every(function(p, dimension) {
          return within[config.dimensions[p].type](d, p, dimension);
        });
      case 'OR':
        return actives.some(function(p, dimension) {
          return within[config.dimensions[p].type](d, p, dimension);
        });
      default:
        throw new Error('Unknown brush predicate ' + config.brushPredicate);
    }
  });

We could filter config.data here first and then perform the selected logic to the filtered config.data.

  // Resolves broken examples for now. They expect to get the full dataset back from empty brushes
  if (actives.length === 0) return config.data;
...
  //  apply outside filters
  //    assuming we have created an appropriate filter function
  let filteredData = filter(config.data);
...
  return filteredData.filter(d => {
    switch (brushGroup.predicate) {
      case 'AND':
        return actives.every(function(p, dimension) {
          return within[config.dimensions[p].type](d, p, dimension);
        });
      case 'OR':
        return actives.some(function(p, dimension) {
          return within[config.dimensions[p].type](d, p, dimension);
        });
      default:
        throw new Error('Unknown brush predicate ' + config.brushPredicate);
    }
  });

Potential Helper Dependency

I discovered search.js that might help with the API for filtering spec.

pinging @gordonwoodhull since you had some interest in the other pull and you have an enormous wealth of knowledge from dc.js

[gordonwoodhull]

Thanks @timelyportfolio! Yes, I've been trying to create a wrapper for this library to interface with dc.js.

What I have tried so far is simply to replace the data whenever there is filtering on the dc.js side... that's how dc.js usually does things.

Performance is okay (for the cars dataset) and it sort of works as long as you brush only parcoords or dc.js, but parcoords goes blank when you brush both, and I haven't figured out why. Another thing I noticed is that the brushed set is not necessarily a subset of the data, so I am setting both. (And quite possibly I am doing it wrong.)

This looks like it may be a better approach; I'll give it a shot!

[timelyportfolio]

@gordonwoodhull wanted to make sure you know that the proof of concept was in the syntagmatic original parallel-coordinates. I plan to implement in parcoord-es by end of weekend if it helps. If you are planning to implement, then I will wait to avoid duplication of effort.

[gordonwoodhull]

Ah, right, I missed that. Most likely you will get to it before me, but I'll let you know if I try anything!

[timelyportfolio]

@BigFatDog @gordonwoodhull I assembled a proof of concept in the filters branch using searchjs as the query DSL. Here is a live example.

Screenshot

Questions

There are a couple of things I don't really like.

1. Currently I only implement for 1D-axes, but I can easily add to the other brushes.
2. parcoords.filter(...) performs the action while parcoods.filters() is the setter/getter for the filters but does not update. I don't know the best way to clear the filters. As of now, parcoords.filter() with no arguments clears the filter, but I don't think this is consistent with the api. parcoords.filters() with no argument gets the filters.
3. I can understand if this is deemed outside of scope for the core parcoords-es library. If so, would there be a way to add as a plugin?

[BigFatDog]

@timelyportfolio
I have gone through your code. Per your questions:

1. We can apply filter function to other brushes after we agree on how to add this function.
2. I have no idea of a better name so far. query is not good because this function changes parcoords 's state other than reads data from parcoords. filterWithQuery is too long
3. Providing this function in a plug-in makes sense. I'll try and come back.

[gordonwoodhull]

Hi @timelyportfolio!

Found a spare moment to experiment with this.

Unfortunately I am still in the AMD era (this fork of parcoords just worked best for me, and seems to be the best maintained). And I couldn't get parcoords.standalone.js to build, probably because I don't have searchjs configured correctly:

$ rollup -c rollup.config.dev.js 

src/index.js → dist/parcoords.standalone.js...
[!] (babel plugin) ReferenceError: Unknown plugin "transform-object-rest-spread" specified in "base" at 1, attempted to resolve relative to "/home/gordon/src/searchjs/src"
../searchjs/src/searchjs.js
ReferenceError: Unknown plugin "transform-object-rest-spread" specified in "base" at 1, attempted to resolve relative to "/home/gordon/src/searchjs/src"
    at error (/usr/local/lib/node_modules/rollup/dist/rollup.js:185:14)
    at Object.error (/usr/local/lib/node_modules/rollup/dist/rollup.js:9407:6)
    at promise.then.previous (/usr/local/lib/node_modules/rollup/dist/rollup.js:9416:32)
    at <anonymous>

I'm a bit lost. Is there a simple way to get it to build so I can test your API with my library?

[BigFatDog]

Solution to the error message: add a .babelrc to your project and then run rollup again:

{
  "presets": ["es2015"],
  "plugins": ["transform-object-rest-spread"]
}

This will help transpile spread operators in es6.

I did try once but found no good solution to integrate search.js in a plug-in style. I plan to try this again this weekend.

I can provide a feature branch quickly so that you can continue with your test. We have something works and then explore the next step.

How do you think?

[gordonwoodhull]

Thanks @BigFatDog! Babel is new to me.

parcoords-es already has .babelrc, of course, so I think the problem is elsewhere. I also tried changing the .babelrc of searchjs/ but no dice.

If it makes any difference, I don't think I really need searchjs for my testing... I will be changing the selection in response to dc.js events

[BigFatDog]

You’re welcome.
dc.js was one of important guides for me to learn d3.js. I knew existence of crossfilter via dc.js

I learn a lot from you.

[gordonwoodhull]

Thanks @BigFatDog, that's nice to hear.

Really all the credit goes to @NickQiZhu and the many dozens of contributors.

For me it's an interesting social experiment, as I never maintained a popular open source library before. I try to help people out and eventually get stuff merged. It's a lot of work and I never have enough time. 😊

[timelyportfolio]

@gordonwoodhull glad you tried it out. I can't get searchjs to work in the proper way either, so I have a hacked solution on my end as well. I'll circle back and try to write a concise way to build and test. I have been using this for a couple of months on one project, and despite the build trouble it has worked very well.

Also, I think it might be helpful to go back and walk through the generic steps necessary to add in a filter with or without searchjs. The reason I used searchjs was to avoid writing a dsl for filtering. I also hoped to present an API that would not require a user to write custom filtering functions for each data type.