Trending crates section

[tkaitchuck]

Currently the homepage of crates.io has sections for:

• New Crates
• Just Updated
• Most Downloaded
• Most Recent Downloads

The new crates and just updated are not very interesting because the crates listed are just whatever happens to have been uploaded before you loaded the page.
The Most Downloaded and Most Recent downloads tend to be fairly static lists, as the list of mainstream widely used crates don't change very rapidly.

This leaves open an area for crates that have recently become popular. Two new sections that could be created without tracking any new statistics are:

• "Newly popular crates" defined as taking the top crates when sorted by: Some constant * Number of downloads in the last 90 days - Number of prior downloads. This would give a list of crates that have just recently become popular. (With a constant probably somewhere between 1 and 10 depending on desired rate of turnover.)
• "Trending crates" defined as taking the top crates when sorted by: Number of downloads in the last 90 days * (Download rate in the last 90 days / Download rate over crate lifetime) which simplifies to ((Downloads in 90 days)^2 / (Total downloads)) * (age in days/90). This would give a list of popular crates whose popularity is increasing.

[ToBinio]

are there any plans to implement this or something similar to this?
or is there some reason not to?

[Turbo87]

are there any plans to implement this or something similar to this?

there are currently no such plans

is there some reason not to?

it would probably make sense to first use the database dump to generate a few of these listings over time to see how useful they are and if the formulas are correct.

[ToBinio]

I tested around with some "algorithms" and the current database dump and wanted to quickly share my results.

firstly the two algorithms defined by tkaitchuck

Newly popular crates

SELECT *
FROM public.crates
      JOIN recent_crate_downloads rcd on crates.id = rcd.crate_id 
ORDER BY 2 * rcd.downloads - crates.downloads DESC
LIMIT 10;

the const-factor makes the absolute number of downloads more important
at a value of value 10 syn pops up
if the value is low it does make the release date more important

top-5:
windows-targets, rustix, winnow, toml_datetime, is-terminal

I also tried instead of using the last 90 days only using the last 30 days

top-5:
colorchoice, anstyle-query, clap_builder, anstream, anstyle

Trending crates

In my opinion, this did not work very well. Basically resulting in the most downloaded crates.

I have changed it so that instead of multiplying it is now dividinging.

SELECT *
FROM public.crates
         JOIN recent_crate_downloads rcd on crates.id = rcd.crate_id
where crates.downloads > 0
ORDER BY ((rcd.downloads * rcd.downloads) / crates.downloads) /
         (greatest(extract('Day' FROM current_date - crates.created_at),1)) DESC
LIMIT 10;

top-5:
clap_builder, winnow, colorchoice, anstyle-query, anstream

I also tried using the last 30 days to calculate the recent downloads. This returned basically the same result just in a different order.

new Crate with the most downloads

because the results felt similar to only looking at newly released crates and sorting by downloads I tried this:

SELECT *
FROM public.crates
where created_at > date((CURRENT_TIMESTAMP - '7 days'::interval))
ORDER BY crates.downloads DESC
LIMIT 10;

this only looks at the last 7 days to give a very recent result.

top-5:
commit_encoding_derive, cynic-introspection, bigerror, nugine-rust-utils, schematic_macros

complex

lastly, I tried creating something myself and ended up with this mess

SELECT *
FROM public.crates
         JOIN last_month_recent_crate_downloads mrcd on crates.id = mrcd.crate_id
         Join last_week_recent_crate_downloads wrcd on crates.id = wrcd.crate_id
         JOIN recent_crate_downloads rcd on crates.id = rcd.crate_id
where (rcd.downloads - mrcd.downloads) > 1000
  and wrcd.downloads > (mrcd.downloads / 4)
ORDER BY mrcd.downloads / (rcd.downloads / 90 * 30) DESC
LIMIT 10;

top-5:
tauri-winres, polars-row, argminmax, autometrics, jrsonnet-stdlib

conclusen

I don't think either of the proposed solutions is perfect but I think any of them would be an improvement to the current state.
the problem in my eyes is that they often prioritize very big crates which often have some connection to another already big and known crate.

I am not sure if this "problem" always exists but all ans crates are connected to clap.
Maybe this test should be redone in something like a month.

maybe replacing 'Most Recent Downloads' with new Crate with the most downloads (name WIP) could be a good start.

Notes

all recent_crate_download tables are created the same as the original just with a different time frame.

I know that the SQL statements are not very good/clear. I just wanted to test/play around a bit.

sry, I didn't expect this to get that long....