Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat: optional indexing and caching of site.pages #104

Merged
merged 1 commit into from
Aug 15, 2022

Conversation

karthik2804
Copy link
Contributor

This is a breaking change with a change required in site.toml to fix.

Initial implementation of having the ability to index content for site.pages for only required templates. Implement the cache proposed in #15 for site.pages to improve performance.

Optional indexing improves the performance of pages that do not depend on site.toml while caching improves the performance of any that uses it.

The templates for which site.pages list is required can be specified in site.toml as

.
.
index_site_pages  = ["main"]

[extra] 
.
.

TODO: Docs.

Some initial performance results for tests on fermyon.com

Optional indexing

This branch

bombardier -c 1 -n 1000 http://localhost:3000/blog/underprovisioning-cloud-services
Bombarding http://localhost:3000/blog/underprovisioning-cloud-services with 1000 request(s) using 1 connection(s)
 1000 / 1000 [=========================================================] 100.00% 76/s 13s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec        77.46      26.13     261.02
  Latency       12.96ms   829.78us    23.75ms
  HTTP codes:
    1xx - 0, 2xx - 1000, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:     3.43MB/s

main branch

bombardier -c 1 -n 1000 http://localhost:3000/blog/underprovisioning-cloud-services
Bombarding http://localhost:3000/blog/underprovisioning-cloud-services with 1000 request(s) using 1 connection(s)
 1000 / 1000 [=========================================================] 100.00% 45/s 21s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec        45.79       4.47      69.55
  Latency       21.85ms     0.96ms    35.97ms
  HTTP codes:
    1xx - 0, 2xx - 1000, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:     2.04MB/s

Caching site.pages

This branch

bombardier -c 1 -n 1000 http://localhost:3000/blog/index
Bombarding http://localhost:3000/blog/index with 1000 request(s) using 1 connection(s)
 1000 / 1000 [=========================================================] 100.00% 51/s 19s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec        52.47      21.15     620.55
  Latency       19.25ms     1.77ms    64.88ms
  HTTP codes:
    1xx - 0, 2xx - 1000, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:     5.07MB/s

main branch

bombardier -c 1 -n 1000 http://localhost:3000/blog/index
Bombarding http://localhost:3000/blog/index with 1000 request(s) using 1 connection(s)
 1000 / 1000 [=========================================================] 100.00% 32/s 30s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec        33.00      13.88      65.60
  Latency       30.29ms     4.16ms    81.09ms
  HTTP codes:
    1xx - 0, 2xx - 1000, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:     3.22MB/s

Note: To test locally on fermyon.com add the following to site.toml when using bartholomew.wasm module from this branch to test caching with either homepage or blog page.

index_site_pages = ["home", "main"]

This was referenced Aug 12, 2022
Copy link
Collaborator

@technosophos technosophos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While I am not licensed to LGTM anymore 😁 , I did review this PR and it looks great.

Copy link
Member

@radu-matei radu-matei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me, thanks for the improvement!

There are a few small changes, but nothing major.

LGTM

src/template.rs Outdated
@@ -20,6 +20,7 @@ pub struct SiteInfo {
pub base_url: Option<String>,
pub about: Option<String>,
pub theme: Option<String>,
index_site_pages: Option<Vec<String>>,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be a public field as well.

src/template.rs Outdated
@@ -186,7 +194,12 @@ impl<'a> Renderer<'a> {
// 3. ???
// 4. Leave it like it is
// 5. Determine that this is out of scope
pages: crate::content::all_pages(self.content_dir.clone(), self.show_unpublished)?,
pages: match index_site_page {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think an if/else statement here might make it simpler.

src/template.rs Outdated
@@ -171,6 +174,11 @@ impl<'a> Renderer<'a> {
request_headers.insert(String::from(key.as_str()), String::from(val));
}

let index_site_page: bool = match &info.index_site_pages {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could probably be a single match statement below instead.

@karthik2804 karthik2804 marked this pull request as ready for review August 13, 2022 00:10
@karthik2804 karthik2804 force-pushed the feat/rendering-performance branch 3 times, most recently from 9f3068a to 384e80a Compare August 15, 2022 18:11
Copy link
Member

@vdice vdice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noticed some spots to update to use index_site_pages which appears to be the config key we've settled on...

docs/content/templates.md Outdated Show resolved Hide resolved
docs/content/configuration.md Outdated Show resolved Hide resolved
docs/content/configuration.md Outdated Show resolved Hide resolved
Signed-off-by: Karthik Ganeshram <karthik.ganeshram@fermyon.com>
Copy link
Member

@vdice vdice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Performance improvements look great! It looks like @radu-matei's feedback has been addressed. LGTM.

Let's remember to point this breaking change out in the forthcoming release notes....

@karthik2804 karthik2804 merged commit 2e621d3 into fermyon:main Aug 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants