GatsbyJS Dynamic Results Pages & Auth

[rorz]

Hi folks,

I've used GatsbyJS many times pre v1, and would love to use the new dynamic page creation for a event management project I'm working on.

This would require searching and querying (w/ GraphQL) an external API to produce a dynamic list of results (events)... like a search results page. We'd have to implement auth (which I know is kinda possible from here: #1100) and payment, also.

Perhaps I'm being too ambitious but there's so many good things about the framework that I don't want to write it off just yet.

Is this possible? I got excited by the GatsbyGram example, but now I'm starting to think you can only generate pages dynamically (that are cached and served statically), instead of actual dynamic pages based on user queries.

[KyleAMathews]

Hey! Gatsby definitely sounds like a good fit for what you're doing.

Gatsby can create statically built pages as well as client-only paths.

For static pages, you'll want to build a source plugin for your API https://www.gatsbyjs.org/docs/create-source-plugin/ and then create pages using that.

For authenticated parts of the site, check out this example site https://github.com/gatsbyjs/gatsby/tree/master/examples/client-only-paths for how to add client only sections of your site.

[rorz]

@KyleAMathews — Thanks so much for taking the time to reply...

I'm definitely going to try this approach out (on a smaller project first), and update this thread in due course :)

[m4rrc0]

Just jumping in to say that it would be great indeed to have a way to use the graphql layer for dynamic fetching. For example, here, the list of events would be created on build like normal so we have the static version of the list, then also have a fetch on the client side after everything is loaded. So if the list has changed since the last build, it gets updated. Even better, the client-side query could be altered by the client for a specific search or depending on its geolocation or...
Could a plugin do that at some point?

[cr101]

I have a similar use case as @MarcCoet and @rorz. We are planning to build a consumer content-heavy website using WordPress as a CMS and the WordPress REST API, however, the existing content on the website will be updated daily and new content will be added daily every few minutes. Also, it looks like gatsby-source-wordpress is a great fit for building websites with static-content that rarely changes.
Would we need to build the app every time we change the content?

[KyleAMathews]

@cr101 you do need to build everytime there's a change but it's pretty quick, 20-60 seconds depending on how many pages and how complex your JavaScript gets.

[KyleAMathews]

@MarcCoet there's not yet a way to do live queries against Gatsby's graphql schema but that'd be a great feature to add

[cr101]

@KyleAMathews Thank you for clarifying that. Since we will have over 10k posts in WordPress the build will take several minutes which means users will not be able to use the website while the build is taking place.

[KyleAMathews]

Builds aren't done by the same process serving the website. You build the site and once it's done you copy the new version of the site over.

[cr101]

The content changes are always made on the live WordPress site which is hosted on the same server where the build of the front-end website occurs.
UPDATE:
I was very excited when I came across Gatsby as it has most of the features we need to build a modern React-powered WordPress web app. However, having to build the app every time the content is changed in WordPress is a big limitation which should be documented.
From my understanding, Gatsby 1.x is best suited for small websites where the content is pretty static (rarely changes).

[sebastienfi]

@cr101 As of yet I haven't included partial build from Wordpress in the gatsby-source-wordpress plugin, which mean that every node is deleted every time the wordpress source plugin is launched, and when the sharp plugin will be here, every image file will be downloaded and processed again (the static and public folder ar deleted before re-build). This functionnality is included in gatsby-source-contentful and my plan is to mimic it so the plugin's architecture look familiar to users.
This is mainly because Wordpress does not - at my knowledge - seem to natively offer a way to query pages, posts and medias created / updated after a given dateTime. My plan is to develop a Wordpress plugin that would add this feature to JSON REST API V2 (in Wordpress Core) (and release it open source). Having this plugin, it is pretty easy in my opinion to download only what's needed from the API, invoking Touch Nodes Method at beginning of onCreateNode() in order to prevent existing nodes to be trashed on, and creating new nodes for newly created Wordpress content, destroying then re-creating nodes for updated contents.
The blog we're currently focusing on publishes 5 to 10 blog posts a day.

About build time, we successfully obtained build time of the site under 7 sec. And yes, each update of the static site is preceeded by a rebuild. My idea is to rebuild the site every hour using a cron job. The job would start by checking if there is new content in order to rebuild only if needed be.

[cr101]

@sebastienfi Just so I understand this correctly.

My idea is to rebuild the site every hour using a cron job. The job would start by checking if there is new content in order to rebuild only if needed be.

Would this rebuild the entire site or just update the nodes for the newly updated content or create new nodes for newly added content and then update/create the respective static pages?

The reason I'm asking is because we are using WordPress as a CMS with the use of the Pods Framework and we have over 10K custom posts in the database. We update the existing content and adding new content every several minutes every day.

[KyleAMathews]

@cr101 I suggest building a prototype to see if Gatsby will meet your needs. Would love help making sure Gatsby scales to bigger sites like yours!

[cr101]

@KyleAMathews Building a prototype is not a bad idea, however, I'd like to first understand if GatsbyJS is a good fit to my scenarios and constraints and equally important to know the drawbacks and whether these can be mitigated.
Just to give you an idea of what some of our WordPress-powered web app requirements are:
A search page which will have:

• a search box that will allow users to search as they type for a specific company, shareholder or director.
• a number of filters to filter the Companies search results by: company status, founded date, closed date, category, number of employees, director gender, etc.

Will it be possible to implement the "search as you type" and the search results filters at runtime?

User login page. Will it be possible to show the user's name in the site header after user logging in and remove it after logging out?
User Settings page. Will users be able to change their password, change their own personal info (i.e email address, location, etc)

And there is still the issue of having to rebuild the entire site every time the content is changed in the WordPress database.

Some folks use WordPress as a blogging platform, however, lots of developers also use WordPress as a CMS which has a massive market share.

[sebastienfi]

@cr101 Everything about search should be ok. For the search, there is 2 scenarios :
1/ doing the search client side if there isn't too many articles.
2/ doing the search server-side, means calling the WP search API Endpoint and parsing the results.
I've done both, the second option is better for larger sites, and some plugins will allow you to tweak the search results.

About the login and stuff, one could implement a OAuth 2 process happening on the client side in order to allow the login request to be run against the WP server. Then you'll have to create a user context, and send this to the requests you send to WP.

I do use WP as a CMS too, but I do use Gatsby to display the site only. The editing happens on the WP Back-end or using Calypso with Jetpack enabled sites.

[cr101]

@sebastienfi Have you used lunr for doing the search and filtering of the search results in the browser?
Perhaps something like that may be faster and more accurate than doing the search server-side since we already have all the data in the client.

[rashidul0405]

@cr101 Would you please share any update on your findings?

[cr101]

@rashidul0405 I didn't use GatsbyJS in the end. Sorry that I can't help you.

[rashidul0405]

@cr101 appreciate your reply. Thanks.

[KyleAMathews]

Thanks for everyone that contributing to this discussion! Clearing out old issues so closing this now.