Tweet Miner is a Simple Project to Scrape Tweets Dynamically from Specific Topics.
It comes with many features, such as authentication using JWT, request validation, unit and integration tests, continuous integration, docker support, API documentation, pagination, etc. For more details about the features, check the list below.
- ES9: latest ECMAScript features
- NoSQL database: MongoDB object data modeling using Mongoose
- Authentication and authorization: using passport
- Validation: request data validation using Joi
- Logging: using winston and morgan
- Testing: unit and integration tests using Jest
- Error handling: centralized error handling mechanism
- API documentation: with swagger-jsdoc and swagger-ui-express
- Process management: advanced production process management using PM2
- Dependency management: with Yarn
- Environment variables: using dotenv and cross-env
- Security: set security HTTP headers using helmet
- Santizing: sanitize request data against xss and query injection
- CORS: Cross-Origin Resource-Sharing enabled using cors
- Compression: gzip compression with compression
- CI: continuous integration with Travis CI
- Docker support
- Code coverage: using coveralls
- Code quality: with Codacy
- Git hooks: with husky and lint-staged
- Linting: with ESLint and Prettier
- Editor config: consistent editor configuration using EditorConfig
Clone the repo:
git clone git@github.com:MRdevX/tweet-miner-api.git
cd tweet-miner-api
Install the dependencies:
yarn install
Set the environment variables:
cp .env.example .env
# open .env and modify the environment variables (if needed)
Running locally:
yarn dev
Running in production:
yarn start
Testing:
# run all tests
yarn test
# run all tests in watch mode
yarn test:watch
# run test coverage
yarn coverage
Docker:
# run docker container in development mode
yarn docker:dev
# run docker container in production mode
yarn docker:prod
# run all tests in a docker container
yarn docker:test
Linting:
# run ESLint
yarn lint
# fix ESLint errors
yarn lint:fix
# run prettier
yarn prettier
# fix prettier errors
yarn prettier:fix
src\
|--config\ # Environment variables and configuration related things
|--controllers\ # Route controllers (controller layer)
|--docs\ # Swagger files
|--middlewares\ # Custom express middlewares
|--models\ # Mongoose models (data layer)
|--routes\ # Routes
|--services\ # Business logic (service layer)
|--utils\ # Utility classes and functions
|--validations\ # Request data validation schemas
|--app.js # Express app
|--index.js # App entry point
To view the list of available APIs and their specifications, run the server and go to http://localhost:3000/v1/docs
in your browser or use the version deployed on Heroku https://tweet-miner-app.herokuapp.com/v1/docs/
. This documentation page is automatically generated using the swagger definitions written as comments in the route files.
List of available routes:
Auth routes:
POST /v1/auth/register
- register
POST /v1/auth/login
- login
POST /v1/auth/refresh-tokens
- refresh auth tokens
POST /v1/auth/forgot-password
- send reset password email
POST /v1/auth/reset-password
- reset password
User routes:
POST /v1/users
- create a user
GET /v1/users
- get all users
GET /v1/users/:userId
- get user
PATCH /v1/users/:userId
- update user
DELETE /v1/users/:userId
- delete user
Topic routes:
POST /v1/topics
- create a topic
GET /v1/topics
- get all topics
GET /v1/topics/:topicId
- get topic
PATCH /v1/topics/:topicId
- update topic
DELETE /v1/topics/:topicId
- delete topic
Tweet routes:
POST /v1/tweets
- create a tweet
GET /v1/tweets
- get all tweets
GET /v1/tweets/:tweetId
- get tweet
PATCH /v1/tweets/:tweetId
- update tweet
DELETE /v1/tweets/:tweetId
- delete tweet
The app has a centralized error handling mechanism.
Controllers should try to catch the errors and forward them to the error handling middleware (by calling next(error)
). For convenience, you can also wrap the controller inside the catchAsync utility wrapper, which forwards the error.
Request data is validated using Joi. Check the documentation for more details on how to write Joi validation schemas.
The validation schemas are defined in the src/validations
directory and are used in the routes by providing them as parameters to the validate
middleware.