Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update mosaic tile generation notebook #12

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

KlaasH
Copy link

@KlaasH KlaasH commented Jul 16, 2024

Proposed Changes:

Applies some changes to mosaic.ipynb for adjustments I made in the course of generating static tiles for WikiWatershed/model-my-watershed#3635.
Most notably, the last section now

  • takes a list of zoom levels and loops over them
  • uses asyncio.Semaphore, via a decorator, to limit how many tile requests will run simultaneously.

I also copied a few variables into the final cell (i.e. COLORMAP and tile_url_template) so it can be run independently of the earlier cells.

Related Issue(s):

Notes:

  • Zoomlevels 0-3 can't be generated with the Lambda tiler because individual tiles take longer than 15 minutes. For those, I used a modified version of the notebook code pointed at a local titiler-mosaicjson server (and limited to concurrency 1. see my comment on #9 for notes on running titiler-mosaicjson locally).
  • The tile downloading cell mostly works if you pull it into a script to run it from the command line, but the
    results = await asyncio.get_event_loop().create_task(download_tiles())
    
    line needs to be changed to
    result = asyncio.run(download_tiles())
    
  • I'm not sure this is the strategy to pursue long-term. Given that the low-zoomlevel tiles can't be generated with Lambda, it's never going to be possible to wrap this up into a simple script that can just be run against the existing tiler endpoint. And if it's going to require local processing and more-extensive supervision anyway, it might make sense to just switch to a "download the files and make a tile pyramid with GDAL tools" strategy. But that's a question for issue AWS 2-6: Evaluate Adding Hybrid Tiling Endpoint #10.

PR Checklist:

  • I have added my changes to the CHANGELOG or a CHANGELOG entry is not required.

Applies some changes to mosaic.ipynb for adjustments I made in the course of
generating static tiles for WikiWatershed/model-my-watershed#3635.
The last section now
- takes a list of zoom levels and loops over them
- uses asyncio.Semaphore, via a decorator, to limit how many tile requests will
  run simultaneously.

I also copied a few variables into the final cell (i.e. COLORMAP and tile_url_template)
so it can be run independently of the of the earlier cells.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants