Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Examples refresh #955

Merged
merged 50 commits into from
Feb 24, 2021
Merged

Examples refresh #955

merged 50 commits into from
Feb 24, 2021

Conversation

cholmes
Copy link
Contributor

@cholmes cholmes commented Jan 27, 2021

Related Issue(s): #821

Proposed Changes:

  1. Moved examples from individual directories into a single /examples folder at the root, and evolved them to be more representative.

PR Checklist:

  • This PR is made against the dev branch (all proposed changes except releases should be against dev, not master).
  • This PR has no breaking changes.
  • I have added my changes to the CHANGELOG.

TODO's

Must have

  • Readme for examples explains all that's going on, compares against best practices
  • update all links to refer to the new examples
  • validate with all the tools, check with best practices, ensure it works with stac browser

Should have (but ok to ship without this...)

  • Link to real data (use absolute url's and just put in some assets on public catalogs) for the 'main' assets (mostly done, but they all use the same assets, would be ideal if they were different, and spread out)
  • Update the extents of catalogs / collections throughout so they reflect the items
  • Update any eo:bands to follow recommendations in Eo clarifications #974 (can be just not show the properties one)
  • Add proj shape and transform, once we get the answer on projection extension best practices? #976
  • Add summaries

After Merge

  • Update the self url's to be dev. Figure out how to handle that with releases and master? Probably just released version, and master 'self' will point at the tagged on. Or Publish new STAC examples #984 would be another route, make them all relative.

@cholmes cholmes self-assigned this Jan 27, 2021
@cholmes
Copy link
Contributor Author

cholmes commented Jan 27, 2021

I'm still playing with the structure, but so far I'm thinking there's one collection with two items in the root examples directory, and then a catalog in the root directory, that links down to at least two different collections, which each link to items. I think I may end up doing one item per (stable) extension. There should likely be more links from the spec itself into the examples. Also thinking that everything here will be 'fictional', except for possible assets - may do absolute links to some known open locations of real satellite data, or maybe try to find sub-100mb images, but I don't really want to have image files in the github repo. I also started https://github.com/stac-utils/stac-examples where we can move 'real' examples - so that should gather production stac items all in one place for people to explore, and then we keep the actual spec to lighter, fictional ones.

After I get the rough structure down I'll need to go back and ensure everything is following as much of our best practices as possible. And make sure we are fully demonstrating all the fields and concepts we articulate in the spec (or as many as is reasonable).

@m-mohr
Copy link
Collaborator

m-mohr commented Jan 29, 2021

Figure out how to make validation work with 'file' schema, as it's not in beta2 release (direct link to the schema?)

For STAC Node validator you can say that it should resolve against a directory (i.e. stac-spec dev branch cloned to your computer), which contains the schemas. See the README.

@cholmes
Copy link
Contributor Author

cholmes commented Jan 29, 2021

For STAC Node validator you can say that it should resolve against a directory (i.e. stac-spec dev branch cloned to your computer), which contains the schemas. See the README.

Ah, sweet - thanks! Will check it out.

@cholmes cholmes added this to the 1.0.0-RC.1 milestone Jan 30, 2021
@cholmes cholmes mentioned this pull request Feb 9, 2021
11 tasks
@cholmes cholmes marked this pull request as ready for review February 9, 2021 05:01
examples/README.md Outdated Show resolved Hide resolved
examples/README.md Outdated Show resolved Hide resolved

#### Catalog Layout

Another important recommendations concerns the [layout of STAC catalogs](../best-practices.md#catalog-layout). This is important
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just add a folder? It's just a single folder, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought it was nice to come to a directory and see key examples right there. To be able to open them easily and compare, instead of having to dig through directories. I think with best practices we'd add a folder for each item, so it's more than a single folder.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, it's a folder per item. So it depends on the outcome of #984? If we deploy and add the assets, then it would make sense to actually follow the best practices. If there's no assets, it doesn't make much sense to add a folder per item.

Nevertheless, I found the structure at the moment a bit confusing because there's no clear separation at first glance (GH schows you the files first and then you see the README)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I agree that if we deploy and add assets it would make sense to follow the best practices, at least in the deployed version. I think that tells a nice story - in github it's a self-contained catalog, for local use, and it's maybe a bit more ok that it's not quite best practices. But publishing would use pystac or another tool to format and publish, which would use the best practices, which is where it's more important to follow all the 'rules' - when you're exposing to the world.

"href": "http://cool-sat.com/catalog/CS3-20160503_132130_04/catalog.json"
"rel": "root",
"href": "./collection.json",
"type": "application/json"
},
{
"rel": "alternate",
"type": "text/html",
"href": "http://cool-sat.com/catalog/CS3-20160503_132130_04/CS3-20160503_132130_04.html"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add the root catalog to STAC Index and then we have real HTML Urls...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe? But I think we should get it set up with the 'publishing' mechanism first, so that it's a final, stable location, and not one we have to update.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the version number update really a problem? Whether search & replace works on 100 or 200 matches in the files don't really care, right? If it makes it work on GH only, maybe it's enough? Otherwise, you need to implement CI and scripts again for deployment.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the version update is really a problem. I was leaning more towards just do it in github, but Matt liked the idea of publishing them to an online location, so I made #984 for him. But I see that as 'nice to have', and the default to me is just to keep it as is in github, and update the version. If we decide not to do the publishing thing then I'm definitely cool to just call it good in github and set the html rel=alternates to stacindex based on this. I just want to be sure that's the route we're taking.

@cholmes
Copy link
Contributor Author

cholmes commented Feb 23, 2021

Ok, I think this is ready to go. Biggest change is the talking about the catalog type. I went back to modify the catalog type best practices, and realized that we already did allow absolute URL's in self-contained catalogs. But I gave the two types each a 'name' and h4 so we could link to them, and clarified things a bit.

@cholmes
Copy link
Contributor Author

cholmes commented Feb 23, 2021

Right before merge I'll change the self url to /dev branch. I do still hope to change it to be a self-contained catalog, with a publish step to put it on stacspec.org somewhere, as in #984 But let's get this in, and I think that's a bit more 'nice to have'.

@m-mohr
Copy link
Collaborator

m-mohr commented Feb 23, 2021

I think I wouldn't change to dev, but instead change to the current version number. It makes links invalid for now, but once we release rc1 they are valid. The issue with dev is that we may forget to update them as this would be a new step to be added to the process.md...

@cholmes
Copy link
Contributor Author

cholmes commented Feb 23, 2021

Ok, that makes sense, as then in rc.1 -> rc.2 or final it'll get picked up in the find and replace. Will change it now, as I think this is ready.

@cholmes
Copy link
Contributor Author

cholmes commented Feb 23, 2021

Ok, updated to v1.0.0-RC.1 for the anticipated tag. Also realized that this wasn't a true 'relative published catalog' as it had more 'self' links than it should. So removed the extra self ones.

@cholmes
Copy link
Contributor Author

cholmes commented Feb 24, 2021

Ok, this is now really, really ready to go (I think). I'd really like to get this merged, along with #989, so that extension stuff is all in and then I can start on #946.

@m-mohr
Copy link
Collaborator

m-mohr commented Feb 24, 2021

I've lost track on open comments so let's merge and see how it looks in reality. We can do more PRs later.

@cholmes
Copy link
Contributor Author

cholmes commented Feb 24, 2021

Sounds great. I think I captured at least some of the open questions in other issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants