Move out documentation website into a separate repo

[mourner]

This PR removes everything related to the GL JS docs website in favor of having an independent mapbox-gl-js-docs repo that pulls mapbox-gl-js in as a submodule.

The rationale behind this is simplifying an incredibly error-prone documentation maintenance process that bit us during releases many times — currently we to manage docs through complicated two-way branch merges with frequent conflicts (master branches into release-<name> which gets merged into publisher-production which gets PRs on its own and then gets merged back to master), with a lot of code and heavy dependencies entangled together.

With the new structure, we will have a simple one-way relation — mapbox-gl-js-docs includes mapbox-gl-js as a submodule for building API documentation from its source but is otherwise completely independent, and updating the website to a new release is just a matter of updating the submodule pin. Additionally, website improvements can happen independently, have their own PRs/issues, CI, processes and conventions, which will simplify GL JS docs maintenance work for our docs team significantly. Maintaining dependencies in GL JS will also get much easier with half of the yarn.lock (7k lines) gone.

One consequence of the change is that contributions to our examples should now target the new repository. This is a small price to pay for all the benefits the split does. Initially I tried to keep examples in the main repo, but it turned out that technically it would be much easier to have them in the new repo. Portions of the style spec docs will have to be maintained there as well.

Do not merge until mapbox-gl-js-docs goes live on docs.mapbox.com.

cc @andrewharvey

[andrewharvey]

Personally I like the current workflow and think it is simple and elegant, as it keeps everything together, and the way you can update the docs now, or bundle doc changes for features which will come into the next release is working very nicely from my point of view, but then again I don't deal with releases which sounds like where the pain is coming from.

• issues - currently there are 81 issues tagged as docs. Will these stay here or have to be move to the new repo? If we leave it here, will PR merges automatically close issues when you merge like they currently do? Whatever we do, I suspect we'll end up with issues mixed between the two repos. If issues are moved to the new repo, people will still report here, even if at first it's not evident it's relating to the docs, so you'll end up with issue which have multiple threads. Sounds much more complicated.

• examples for new features, currently you could just include them with the feature PR, and they go live automatically upon the new release. Now we'll need to have a corresponding PR on the docs side, and then track which release that's for to control when it gets merged? This makes it harder to review and work on as everything is split.

• on the mapbox-gl-js-docs side, sometimes you'd be writing examples for features already released (can merge now) or for examples for features in the next release. We'd want to include some contributing information on how to deal with those two scenarios.

• loosing the history of the docs in the new repo is a shame.

[mourner]

I don't deal with releases which sounds like where the pain is coming from.

Yeah, it has been a huge pain, and also a source of significant release delays. It's the reason why pretty much half of the history for the docs folder looks like this:

issues - currently there are 81 issues tagged as docs. Will these stay here or have to be move to the new repo? If we leave it here, will PR merges automatically close issues when you merge like they currently do?

I believe issues related to API docs will stay here, while issues related to the website itself (like a menu not working) can be opened there. We can transfer issues freely between the two — GitHub now makes it easy. Most of the docs fixes will go to this repo because they will be targeting JSDoc comments, and close corresponding issues when the PR is merged.

examples for new features, currently you could just include them with the feature PR, and they go live automatically upon the new release. Now we'll need to have a corresponding PR on the docs side, and then track which release that's for to control when it gets merged? This makes it harder to review and work on as everything is split.

That's true, although with the split, retaining examples in this repo also means it's more difficult to update and add examples between releases, so given these two use cases, I chose the latter as more important and also simpler to implement (since otherwise we'd have to decouple example content from the website architecture like Batfish templates, while keeping code & metadata together). Would you prefer adding/updating examples only on releases but keeping them together with the code? (For important fixes, we could always cherry-pick into a release branch and then update the pin in the docs).

on the mapbox-gl-js-docs side, sometimes you'd be writing examples for features already released (can merge now) or for examples for features in the next release. We'd want to include some contributing information on how to deal with those two scenarios.

Sure.

loosing the history of the docs in the new repo is a shame.

I tried migrating the history to the new repo at first (with git filter-branch history rewrite), but it didn't prove very useful:

• Past commits before setting up mapbox-gl-js as a submodule would all be in a broken state you can't build docs from. Meaning you can't use things like git bisect.
• The history looks very messy, as I shown earlier — a ton of merge commits and also duplicates due to cherry-picks.
• For git blame, we could just browse pre-split mapbox-gl-js (adding a link to it in the readme) if the history you're interested in goes past the split.
• If you're concerned about accurate contribution counts in GitHub, migrating the history would mean you get duplicate entries for everything that went to the docs. I think retaining history is important when you merge repos (since you get rid of the repo you merged in), but it makes sense to start over when you're splitting them.

Thanks for the thorough feedback @andrewharvey! This architecture isn't set in stone yet, so we'll be very happy to hear your suggestions and make the new system work well for everyone involved.

[andrewharvey]

Yeah, it has been a huge pain, and also a source of significant release delays. It's the reason why pretty much half of the history for the docs folder looks like this:

Yeah I saw that, so I'll jump onboard to the changes on the basis that this will ease issues for doing releases.

We can transfer issues freely between the two — GitHub now makes it easy.

Oh right, i didn't realise you can transfer issues. That should make this less off a mess.

Would you prefer adding/updating examples only on releases but keeping them together with the code?

No I think it's better to just move examples across to the new repo, if the example depends on features from the next release, the PR can be marked as such, and then merged once the release happens, all other PRs that don't depend on the next release can be merged whenever.

If you're concerned about accurate contribution counts in GitHub, migrating the history would mean you get duplicate entries for everything that went to the docs. I think retaining history is important when you merge repos (since you get rid of the repo you merged in), but it makes sense to start over when you're splitting them.

Okay, fair enough. Whatever works best.

[asheemmamoowala]

Would you prefer adding/updating examples only on releases but keeping them together with the code?

No I think it's better to just move examples across to the new repo, if the example depends on features from the next release, the PR can be marked as such, and then merged once the re

From looking at the current setup of the examples pages - there is an html page with the source and a js page with the metadata. Keeping the examples in this repo would either split the examples source and meta data pages across two repos, or require maintaining both here at the risk of keeping the batfish and other dependencies. @mourner I think you've already mentioned this in our previous discussion. Either way - contributing an example in this repo would not directly publish updated docs. Given that link is being broken now, having the examples in the docs repo with direct publish availability seems like a worthy reason to move.

Separating the examples is harder for local debugging and testing too, but it should be fairly easy for developers to work on the docs repo and pin the gl-js submodule to their branch for local testing. This type of workflow has been common on the team when porting features to the mapbox-gl-native repo.

[andrewharvey]

Separating the examples is harder for local debugging and testing too, but it should be fairly easy for developers to work on the docs repo and pin the gl-js submodule to their branch for local testing. This type of workflow has been common on the team when porting features to the mapbox-gl-native repo.

That's true, I'll often have some local changes to gl-js and then start-docs to test out the example with those local changes. Is it possible to have that gl-js submodule from docs point to a local gl-js repo to support that use case?

[colleenmcginnis]

@mourner this looks ok to me -- not sure if you want to tag anyone else who's familiar with non-docs files to take a look and confirm all non-docs workflows are not affected.

[asheemmamoowala]

🎉

[andrewharvey]

I believe issues related to API docs will stay here, while issues related to the website itself (like a menu not working) can be opened there. We can transfer issues freely between the two — GitHub now makes it easy. Most of the docs fixes will go to this repo because they will be targeting JSDoc comments, and close corresponding issues when the PR is merged.

Should we go through and move over issues that are related to the docs website/workflow that are currently here to mapbox-gl-js-docs?

[colleenmcginnis]

Should we go through and move over issues that are related to the docs website/workflow that are currently here to mapbox-gl-js-docs?

Yes @andrewharvey! I'm going to take the lead on this.

[korywka]

@mourner I think moving controls to separate repo also worth consideration 😉