Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(counter): exluding stopwords and occurences <= 3 #15

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

MarkNygaard
Copy link

Summary

This pull request optimizes the commonWords function by introducing filtering for common stop words and adjusting the word counting mechanism to reduce noise. The key changes include:

  1. Stop Words Filtering: Excludes common stop words such as "the," "and," "to," etc., to focus on more meaningful words.
  2. Count Filtering: Only includes words that appear more than 3 times to further reduce noise and highlight significant words.

Detailed Changes

  • Stop Words Exclusion:

    • Added an array excludedWords containing the most common English stop words.
    • Modified the normalizedWords map function to filter out words present in excludedWords.
  • Count Threshold:

    • Introduced a filter step to only include words with a count greater than 3 in the final output.

Benefits

  • Improved Relevance: By excluding common stop words, the function now highlights more relevant and meaningful words in the text.
  • Noise Reduction: The additional filter for word count ensures that only frequently occurring words are included, reducing the noise from less significant words.
  • Enhanced Readability: The output is more focused and easier to interpret, providing better insights from the text analysis.

Copy link

netlify bot commented Jun 7, 2024

Deploy Preview for datocms-plugin-word-counter ready!

Name Link
🔨 Latest commit 5afa952
🔍 Latest deploy log https://app.netlify.com/sites/datocms-plugin-word-counter/deploys/666342055e9dd700084d4e50
😎 Deploy Preview https://deploy-preview-15--datocms-plugin-word-counter.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant