How to disable prefetching of pages?

[i8ramin]

Hello. Just wondering how one goes about disabling the prefetching features of Gatsby? Is it possible to opt-in to prefetching vs having it enabled for all links by default?

[KyleAMathews]

All things are possible :-)

What's your reasoning?

[i8ramin]

Mainly want to have more control over which assets are being loaded. Not clear how the current mechanism works. Does it crawl the entire site and load all data required to render every page?

For example, I've noticed on the main gatsbyjs site that when you load up the homepage, it also loads an additional 50+ JS bundles, ranging anywhere from 200B to 25KB (totaling about 200KB). The homepage JS seems to be around 30-40kb, but then another additional few hundred KB of data is sent down as well. I would imagine that not everyone would want this behavior, or would at least want to be able to control it somehow.

Maybe a better way to ask this is, is it possible to lazy load bundles for certain pages?

[KyleAMathews]

is it possible to lazy load bundles for certain pages?

It'd be good to have an API for this. There needs to be more discussion about the right API for this. One thing I'd like to add support for example is being able to pass in the order of preloading. Gatsby uses a heuristic for determining that right now but it'd be possible from analytics to determine what pages are viewed after a landing page and preload those first.

[KyleAMathews]

So instead of disabling lazy loading, Gatsby could say internally, "don't preload any page that doesn't have more than a 5% chance of being loaded within the next 10 seconds."

In general, I'd much prefer data/algorithmic driven decisions vs. making programmers guess.

[markmichon]

I nearly posted this in the v2 discussion, but this is related so...

I've been toying with the idea of ways to better handle the prefetching pages. If I understand the system, currently gatsby pulls in data for all linked pages. From a perceived performance standpoint, all good. The page loads fast, then the browser pulls in additional data to prevent future delays. The downside is that we're still consuming bandwidth, even if a user never navigates or thinks about navigating to the page. Example, Reactjs.org's all posts page. Super fast load, but massive bandwidth numbers.

I can foresee a solution where Gatsby only prefetches content for pages once the link is in view. This combined with API hooks for load priority as mentioned above, and users could potentially poll their analytic source of choice at build, configure order, and prefetch the minimum viable bundles.

No clue where to start, but if anyone can point me in the direction (@KyleAMathews) I'll see if I can wrap my head around it.

This might be better served as a plugin (if it's even possible in plugin form). Thoughts?

[KyleAMathews]

I can foresee a solution where Gatsby only prefetches content for pages once the link is in view

This is a really good idea. It'd be very cheap to implement with IntersectionOberver and a link being in view is an excellent heuristic for when to trigger prefetching.

I just did this for gatsby-image for image lazy loading and it was super easy. @markmichon any interest in doing this for gatsby-link? We'd want something similar to gatsby-image, only trigger preloading if the link is in view. This is a really cheap perf/bandwidth upgrade for modern browsers supporting IntersectionObserver.

On hooking up to analytics, I don't think the API will be super hard. The real work will be connecting to stuff like Google Analytics and creating the model for predicting preloading. Happy to take proposals for this and PR prototypes. One idea for a API would be just that plugins could disable the automatic preloading and instead they gets events for when links get mounted and route changes and the plugin completely handles preloading.

[markmichon]

Yeah I'm down for tackling it. Interaction observer was where my head was going. I didn't realize gatsby-link was dictating the loading, but rather something with the webpack setup was. Is it the __loader part handling this?

As far as the API, I wouldn't even focus on the "predicting" side for Gatsby, but rather just provide the hook and let plugin developers (or official plugins) handle the outside connections.

[KyleAMathews]

Yeah I'm down for tackling it

Sweet!

I didn't realize gatsby-link was dictating the loading

It doesn't. It just signals that it was mounted. These two files are what controls prefetching https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby/cache-dir/prefetcher.js and https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby/cache-dir/loader.js

I wouldn't even focus on the "predicting" side for Gatsby

Right, this is what I meant. A plugin would only have to disable the automatic preloading and replace it with its own logic, most likely triggered by listening to route/link mount events.

[i8ramin]

I dig the idea of prefetching only visible links. However, I still think it would be nice to be able to disable the feature if desired. I don't have a full grasp of the code so I can't speak to how easy/hard it would be to make it configurable, but if it is an easy thing to add, could be helpful in certain cases.

[lourd]

Hey y'all. I've got a use case that's related to this, but the inverse. The nav menu on my gatsby site is full-screen and hidden to start with. I'm not sure which of those attributes is causing the intersection observer not to pickup when it become visible, but in either case, none of the other pages on my site are being preloaded. I'd like to get around that by being explicit, with something like a preload prop that's a boolean.

Adding explicit preloading, and by extension, explicit disabling of preloading, looks like it would only be one additional prop and a couple lines in gatsby-link. Preloading would be enabled here and disabled here.

Is there support for re-opening discussion and adding this feature?

[KyleAMathews]

@lourd hmmm we should debug why IntersectionObserver isn't working there.

I'd rather not add a preload prop as that complicates using Gatsby unnecessarily.

[lourd]

It looks like the routes aren't being preloaded on Safari either 🤔 and they should, since it doesn't have IntersectionObserver support yet....

🤦‍♂️

Turns out the intersection observer was working fine. The routes weren't loading because of a trailing slashes mismatch. Adding gatsby-plugin-remove-trailing-slashes fixed the issue. Carry on, apologies for the noise.

[KyleAMathews]

Hmm weird about Safari. Just tested gatsbyjs.org on Safari and it's preloading as expected

[lourd]

Yes, sorry if I wasn't clear. The preloading is working fine. Safari eagerly pre-loads everything as expected. My pages weren't preloading because the link paths I specified in my content did not have trailing slashes, but the page paths did in gatsby's page manifest.

[matyunya]

For those still interested in disabling prefetching, I suggest putting it in dotenv file as follows and fixing this couple lines in node_modules/gatsby/cache-dir/loader.js (line 30, as of v1.9.214):

and then line 242

Then add to your .env.production in root dir
NO_PREFETCH=true

Hope this might help someone.

[matyunya]

My argument to disable prefetch is that when you lazy load images with gatsby-image wrapped in gatsby-link you might not want those extra network requests, especially considering slow networks.

[stuckj]

I have a use case for selectively disabling preloading. I have a link in a menu that goes to a page that requires basic auth. If you mouse over the link a basic auth prompt pops up as gatsby tries to pre-load it. Seems like a perfect case for being able to turn pre-loading off on a specific link.