Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

queryRenderFeatures continued [not ready] #2224

Merged
merged 48 commits into from
Mar 24, 2016

Commits on Mar 24, 2016

  1. Configuration menu
    Copy the full SHA
    e34197e View commit details
    Browse the repository at this point in the history
  2. index collision boxes with a grid instead of rtree

    The combined running time of placeCollisionFeature and
    insertCollisionFeature is now 2/3rds of what it used to be.
    
    The grid index divids a square into n x n cells. When a bbox is
    inserted, it is added to the array of every cell it intersects. When
    querying, look in all the cells that intersect the query box and then
    compare all the individual bboxes contained in that cell against the
    query box.
    
    This kind of index is faster in this specific case because it's
    characteristics better match the work we're using it for.
    
    Queries can take 1.3x as long (slower) with the grid index.
    But insertions often take < 0.1x as long with the grid index.
    Our collision detection code does a lot of insertions so this is a
    worthwhile tradeoff.
    
    The grid index also takes advantage of the fact that label boxes are
    fairly evenly sized and evenly distributed.
    
    The grid index can also be serialized to an ArrayBuffer. The ArrayBuffer
    version can be queried without deserializing all the individual objects
    first. This is not currently used.
    ansis committed Mar 24, 2016
    Configuration menu
    Copy the full SHA
    b62895a View commit details
    Browse the repository at this point in the history
  3. store CollisionBoxes in an ArrayBuffer

    This reduces the long-term memory usage of a tile by ~500KB. Saving
    memory is significant because it lets us cache more parsed tiles.
    
    This also speeds up CollisionFeature creation by about 60%.
    
    It's hard to measure, but this probably saves us a bit of gc time.
    
    Using an ArrayBuffer to store the data makes it potentially
    transferrable, which moves us closer to eliminating worker state.
    
    ---
    
    This also adds StructArray, a pretty fast and convenient way of adding
    conceptual objects to ArrayBuffers.
    
    var BoxArray = createStructArrayType([
        { type: 'Int16', name: 'x1' },
        { type: 'Int16', name: 'y1' },
        { type: 'Int16', name: 'x2' },
        { type: 'Int16', name: 'y2' },
        { type: 'Float32', name: 'scale' },
    ]);
    
    var boxes = new BoxArray();
    boxes.emplaceBack(-2, -1, 2, 1, 1.23);
    boxes.emplaceBack(-3, -1, 3, 1, 1.23);
    var box = boxes.at(0);
    assert(box.x2 - box.x1 === 4);
    box._setIndex(1); // avoid creating a new box object
    assert(box.x2 - box.x1 === 6);
    
    How fast is it compared to regular objects?
    Adding features with emplaceBack is faster.
    Accessing and setting individual components is a tiny bit slower.
    ansis committed Mar 24, 2016
    Configuration menu
    Copy the full SHA
    11c8bb5 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    9b4f1b3 View commit details
    Browse the repository at this point in the history
  5. optimize queryRenderedFeatures

    ansis committed Mar 24, 2016
    Configuration menu
    Copy the full SHA
    3e45db3 View commit details
    Browse the repository at this point in the history
  6. use grid index instead of rtree for querying features

    Why?
    Both indexes have comparable querying performance and Grid is
    transferrable. Insertion is faster with Grid than rbush but that's not
    the main benefit.
    
    Point queries are really fast for both. Each is sometimes faster than
    the other. Therre is no clear winner.
    
    Large, full-screen bbox queries are 10% slower with Grid than rbush but
    difference is not significant (0.2% of FeatureTree.query).
    
    comparisons were done with mapbox-streets-v6
    ansis committed Mar 24, 2016
    Configuration menu
    Copy the full SHA
    16655f7 View commit details
    Browse the repository at this point in the history
  7. move queryRenderedFeatures to the main thread

    - it's fast
    - this helps avoid worker postMessage clone costs
    - this prevents busy workers from delaying the results
    - this is a step towards lazily constructing GeoJSON and eliminating
      `includeGeometry`
    ansis committed Mar 24, 2016
    Configuration menu
    Copy the full SHA
    9bd9b0a View commit details
    Browse the repository at this point in the history
  8. index each line/ring of a geometry separately

    This improves performance for large multipolygons and multilines.
    mapbox-streets-v7 collects features with identical properties and
    combines into a single giant feature. This saves space and speeds up
    parsing but it also makes indexing by feature bbox useless.
    ansis committed Mar 24, 2016
    Configuration menu
    Copy the full SHA
    571a8e7 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    dd4f2a0 View commit details
    Browse the repository at this point in the history
  10. remove includeGeometry and lazily build geojson geometry

    The code for converting to geojson is copied from
    https://github.com/mapbox/vector-tile-js
    0f73eb2c417979601be973e046e169c74147ca7
    ansis committed Mar 24, 2016
    Configuration menu
    Copy the full SHA
    e72a467 View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    3da7cd4 View commit details
    Browse the repository at this point in the history
  12. move querySourceFeatures to main thread

    It's much faster to deserialize straight from pbf than to deserialize
    and reserialize as GeoJSON. This also lets us lazily load geometries to
    make it even faster for cases where not all geometries are used.
    
    benchmark with buildings in mapbox-streets-v7
    old (with stringify added)          85ms
    new                                 1.5 ms
    new with all geometries loaded      15ms
    
    benchmark with buildings in mapbox-streets-v6
    old                                 500ms
    old (with stringify added)          150ms
    new                                 3ms
    new with all geometries loaded      15ms
    ansis committed Mar 24, 2016
    Configuration menu
    Copy the full SHA
    845e9d2 View commit details
    Browse the repository at this point in the history
  13. remove StuctArray._struct

    ansis committed Mar 24, 2016
    Configuration menu
    Copy the full SHA
    0bbd180 View commit details
    Browse the repository at this point in the history
  14. Configuration menu
    Copy the full SHA
    73a6e57 View commit details
    Browse the repository at this point in the history
  15. add async version of queryRenderedFeatures

    Despite being potentially parallelized, this often takes longer than the
    synchronous verion because the features have to be parsed from the pbf
    twice instead of once.
    
    The advantage of the async version is that it spends less time in the
    main thread which can keep the everything more responsive. It still
    takes some time to parse all the features from the vector tile into
    geojson (~15ms for a full screen query in a dense area).
    
    Parsing the pbf on the main thread is faster than parsing json.
    Parsing the pbf on the main thread also lets us lazily load the geometry
    and avoid implementing the `includeGeometry` option.
    ansis committed Mar 24, 2016
    Configuration menu
    Copy the full SHA
    abc7b93 View commit details
    Browse the repository at this point in the history
  16. fix query geojson layers

    ansis committed Mar 24, 2016
    Configuration menu
    Copy the full SHA
    2b971f5 View commit details
    Browse the repository at this point in the history
  17. optimize polygonIntersectsBufferedMultiLine

    for when polygon is just a point.
    ansis committed Mar 24, 2016
    Configuration menu
    Copy the full SHA
    8dc1d43 View commit details
    Browse the repository at this point in the history
  18. Configuration menu
    Copy the full SHA
    1227aa0 View commit details
    Browse the repository at this point in the history
  19. Configuration menu
    Copy the full SHA
    e89fe1c View commit details
    Browse the repository at this point in the history
  20. only create Int32Array for non-empty cells

    This significantly reduces memory usage for CollisionTiles that aren't
    used, or are barely used.
    ansis committed Mar 24, 2016
    Configuration menu
    Copy the full SHA
    c84642f View commit details
    Browse the repository at this point in the history
  21. Configuration menu
    Copy the full SHA
    a92d4e0 View commit details
    Browse the repository at this point in the history
  22. fix query tests

    ansis committed Mar 24, 2016
    Configuration menu
    Copy the full SHA
    a31b379 View commit details
    Browse the repository at this point in the history
  23. Configuration menu
    Copy the full SHA
    14df0f3 View commit details
    Browse the repository at this point in the history
  24. Configuration menu
    Copy the full SHA
    cfdda7b View commit details
    Browse the repository at this point in the history
  25. Configuration menu
    Copy the full SHA
    5f60ecc View commit details
    Browse the repository at this point in the history
  26. Configuration menu
    Copy the full SHA
    066462e View commit details
    Browse the repository at this point in the history
  27. Configuration menu
    Copy the full SHA
    099ec01 View commit details
    Browse the repository at this point in the history
  28. fix tests for queryRenderedFeatures

    remove all FeatureTree tests. FeatureTree is covered by the query render tests.
    It's really hard to construct and verify test cases without visual help.
    Whenever the FeatureTree tests break its hard to tell if its a broken
    test or a broken implementation. It's way easier to create good query
    tests.
    ansis committed Mar 24, 2016
    Configuration menu
    Copy the full SHA
    1625f6d View commit details
    Browse the repository at this point in the history
  29. Configuration menu
    Copy the full SHA
    d76b23f View commit details
    Browse the repository at this point in the history
  30. Configuration menu
    Copy the full SHA
    0fdd868 View commit details
    Browse the repository at this point in the history
  31. Configuration menu
    Copy the full SHA
    f8aeecf View commit details
    Browse the repository at this point in the history
  32. remove dead code

    ansis committed Mar 24, 2016
    Configuration menu
    Copy the full SHA
    1594938 View commit details
    Browse the repository at this point in the history
  33. Configuration menu
    Copy the full SHA
    e56a49e View commit details
    Browse the repository at this point in the history
  34. rename layers property from layer to layers

    and accept only an array of strings.
    
    fix #2230
    ansis committed Mar 24, 2016
    Configuration menu
    Copy the full SHA
    dfb96c8 View commit details
    Browse the repository at this point in the history
  35. Configuration menu
    Copy the full SHA
    3164a64 View commit details
    Browse the repository at this point in the history
  36. Configuration menu
    Copy the full SHA
    b6fe1c0 View commit details
    Browse the repository at this point in the history
  37. Configuration menu
    Copy the full SHA
    64a77b9 View commit details
    Browse the repository at this point in the history
  38. Configuration menu
    Copy the full SHA
    41c4b70 View commit details
    Browse the repository at this point in the history
  39. Configuration menu
    Copy the full SHA
    20cc526 View commit details
    Browse the repository at this point in the history
  40. add query benchmarks

    ansis committed Mar 24, 2016
    Configuration menu
    Copy the full SHA
    956a6e9 View commit details
    Browse the repository at this point in the history
  41. Configuration menu
    Copy the full SHA
    d886a2e View commit details
    Browse the repository at this point in the history
  42. Configuration menu
    Copy the full SHA
    25df374 View commit details
    Browse the repository at this point in the history
  43. get rid of Struct._setIndex

    It's faster, but not fast enough to be worth it.
    ansis committed Mar 24, 2016
    1 Configuration menu
    Copy the full SHA
    afa28ea View commit details
    Browse the repository at this point in the history
  44. Configuration menu
    Copy the full SHA
    b1517ef View commit details
    Browse the repository at this point in the history
  45. Configuration menu
    Copy the full SHA
    f96e807 View commit details
    Browse the repository at this point in the history
  46. remove queryRenderedFeaturesAsync

    let's try to get by with just queryRenderedFeatures
    ansis committed Mar 24, 2016
    Configuration menu
    Copy the full SHA
    e79a7ec View commit details
    Browse the repository at this point in the history
  47. Configuration menu
    Copy the full SHA
    7531605 View commit details
    Browse the repository at this point in the history
  48. rename FeatureTree to FeatureIndex

    and split intersection tests into a separate file
    ansis committed Mar 24, 2016
    Configuration menu
    Copy the full SHA
    056fbc0 View commit details
    Browse the repository at this point in the history