Skip to content

Commit

Permalink
Retention policy (#73)
Browse files Browse the repository at this point in the history
* added settingsoptions for retention policy and their labels in locales

* added new settigs for retention policy

* changed minimum deletion period to 1

* started cronjob creation - deleting refunded travel after x days defined in the settings

* some .json changes

* moved cron job instanciation to server.js by importing the method 'retentionPolicy' from 'retentionPolicy.ts + removed import in app.ts

* created retentionPolicy Object containing the settings for the retention policy

* added basic functionallity of deleting old, not updated reports/travels and sending a notification mail -- some changes required

* added default settongs for retention policy

* some changes in common/forms/*.json

* setting labels for setting form

* added condition for retention policy settings: min -1

* changed comment on CronJob

* cleared code, created new method for reducing code, added some comments

* added logic for only send notification mals for none historic versions of reports + changed deletion logic that historic versions of reports are deleted together with their current version, moved parts of the code into own methods to reduce code

* added email template and necessary information incl. locales

* changed instanciation in settings.ts, removed the Record<>

* cleaned code, removed duplicate code by restructuring methods

* added integer validation on values of retentionPolicy and changed infinite value from -1 to 0

* added Type schemaNames to types containing the uppercase Names of the Report Schemas -fixed some types in retentionpolicy.ts

* added description for retention policy settings incl. locales: "setting 0 will prevent deletion/mail"

* added name of report in the locales for notification mail - some changes regarding the days until deletion in notifcation mails

* removed typo

* changed type of retentionSettings and renamed it to retentionPolic; added function getPolicyElements that returns deletions and notifications array; removed some logs and try .. catch;added lean() to User.findOne

* changed log for deleted reports

* mails are now only be sent on the day exact x days before deletion - not every day until deletion - calculation of real Days until deletion is no longer nessesary -> removed

* removed another log

* changed labels

* settings.json updated

* added .lean() on the report request + fixed the type error

* added option of custom description translation key that could be set in the schema object

* removed duplicate translation

* removed labelstr option for description

* added missing descripton translation key

* set default retentions to 0 and mail to 7; added condition that mails are only sent if retentions != 0
  • Loading branch information
sgempi authored Jul 9, 2024
1 parent 670fb3c commit 072ed8c
Show file tree
Hide file tree
Showing 9 changed files with 281 additions and 4 deletions.
7 changes: 7 additions & 0 deletions backend/data/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -82,5 +82,12 @@
"expenseReport": false,
"healthCareCost": false
},
"retentionPolicy": {
"deleteRefundedAfterXDays": 0,
"deleteApprovedTravelAfterXDaysUnused": 0,
"deleteInWorkReportsAfterXDaysUnused": 0,
"mailXDaysBeforeDeletion": 7
},

"version": "1.1.3"
}
27 changes: 26 additions & 1 deletion backend/models/settings.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import {
ExpenseReportState,
HealthCareCostState,
ReportType,
RetentionType,
Settings,
TravelState,
_id,
Expand All @@ -13,6 +14,7 @@ import {
expenseReportStates,
healthCareCostStates,
reportTypes,
retention,
travelStates
} from '../../common/types.js'
import '../db.js'
Expand Down Expand Up @@ -55,6 +57,28 @@ for (const report of reportTypes) {
disableReportType[report] = { type: Boolean, required: true }
}

const retentionPolicy = {} as {
[key in RetentionType]: {
type: NumberConstructor
min: number
required: true
validate: { validator: any; message: string }
description: string
}
}
for (const policy of retention) {
retentionPolicy[policy] = {
type: Number,
min: 0,
required: true,
validate: {
validator: Number.isInteger,
message: 'Must be Integer'
},
description: 'description.' + policy
}
}

export const settingsSchema = new Schema<Settings>({
allowSpouseRefund: { type: Boolean, required: true },
allowTravelApplicationForThePast: { type: Boolean, required: true },
Expand All @@ -79,7 +103,8 @@ export const settingsSchema = new Schema<Settings>({
stateColors: { type: stateColors, required: true },
accessIcons: { type: accessIcons, required: true },
version: { type: String, required: true, hide: true },
migrateFrom: { type: String, hide: true }
migrateFrom: { type: String, hide: true },
retentionPolicy: { type: retentionPolicy, required: true }
})

export type SettingsSchema = InferSchemaType<typeof settingsSchema>
Expand Down
5 changes: 5 additions & 0 deletions backend/models/vueformGenerator.ts
Original file line number Diff line number Diff line change
Expand Up @@ -52,11 +52,16 @@ function mapSchemaTypeToVueformElement(
vueformElement['rules'].push('max:' + schemaType.max)
}

if (schemaType.description) {
vueformElement['description'] = translate(schemaType.description, language)
}

if (schemaType.label) {
vueformElement['label'] = translate(schemaType.label, language)
} else if (labelStr) {
vueformElement['label'] = translate('labels.' + labelStr, language)
}

if (schemaType.info) {
vueformElement['info'] = translate(schemaType.info, language)
}
Expand Down
153 changes: 153 additions & 0 deletions backend/retentionpolicy.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
import { model } from 'mongoose'
import {
AnyState,
ExpenseReport as IExpenseReport,
HealthCareCost as IHealthCareCost,
Travel as ITravel,
Locale,
ReportType,
RetentionType,
reportIsHealthCareCost,
reportIsTravel,
schemaNames
} from '../common/types.js'
import i18n from './i18n.js'
import { sendMail } from './mail/mail.js'
import Settings from './models/settings.js'
import User from './models/user.js'

async function getForRetentionPolicy(schema: schemaNames, date: Date, state: AnyState, startDate?: Date) {
let res: Array<ITravel | IExpenseReport | IHealthCareCost>
if (startDate) {
res = await model(schema)
.find({ state: state, updatedAt: { $gte: startDate, $lt: date }, historic: false })
.lean()
} else {
res = await model(schema)
.find({ state: state, updatedAt: { $lt: date }, historic: false })
.lean()
}

return res
}

function getDateThreshold(days: number) {
let dateThreshold = new Date()
dateThreshold.setHours(0, 0, 0, 0)
dateThreshold.setDate(dateThreshold.getDate() - days)
return dateThreshold
}

async function getPolicyElements(retentionPolicy: { [key in RetentionType]: number }) {
const elements: { schema: schemaNames; state: AnyState; deletionPeriod: number }[] = [
{ schema: 'Travel', state: 'refunded', deletionPeriod: retentionPolicy.deleteRefundedAfterXDays },
{ schema: 'Travel', state: 'approved', deletionPeriod: retentionPolicy.deleteApprovedTravelAfterXDaysUnused },
{ schema: 'ExpenseReport', state: 'refunded', deletionPeriod: retentionPolicy.deleteRefundedAfterXDays },
{ schema: 'ExpenseReport', state: 'inWork', deletionPeriod: retentionPolicy.deleteInWorkReportsAfterXDaysUnused },
{ schema: 'HealthCareCost', state: 'refunded', deletionPeriod: retentionPolicy.deleteRefundedAfterXDays },
{ schema: 'HealthCareCost', state: 'inWork', deletionPeriod: retentionPolicy.deleteInWorkReportsAfterXDaysUnused }
]
return elements
}

async function triggerDeletion(retentionPolicy: { [key in RetentionType]: number }) {
let deletions = await getPolicyElements(retentionPolicy)
for (let i = 0; i < deletions.length; i++) {
if (deletions[i].deletionPeriod > 0) {
let date = getDateThreshold(deletions[i].deletionPeriod)
let result = await getForRetentionPolicy(deletions[i].schema, date, deletions[i].state)
if (result.length > 0) {
await deleteAny(result, deletions[i].schema)
}
}
}
}

async function deleteAny(reports: Array<ITravel | IExpenseReport | IHealthCareCost>, schema: schemaNames) {
let result: any
for (let i = 0; i < reports.length; i++) {
result = await model(schema).deleteOne({ _id: reports[i]._id })
if (result && result.deletedCount == 1) {
console.log(
`Deleted ${schema} from owner ${reports[i].owner.name.givenName} ${reports[i].owner.name.familyName} with name ${reports[i].name}.`
)
}
}
}

async function notificationMailForDeletions(retentionPolicy: { [key in RetentionType]: number }) {
let notifications = await getPolicyElements(retentionPolicy)
if (retentionPolicy.mailXDaysBeforeDeletion > 0) {
for (let i = 0; i < notifications.length; i++) {
if (notifications[i].deletionPeriod != 0) {
let daysUntilDeletionTemp =
retentionPolicy.mailXDaysBeforeDeletion < notifications[i].deletionPeriod
? retentionPolicy.mailXDaysBeforeDeletion
: notifications[i].deletionPeriod
let date = await getDateThreshold(notifications[i].deletionPeriod - daysUntilDeletionTemp)
let startDate = new Date(date)
startDate.setDate(startDate.getDate() - 1)
let result = await getForRetentionPolicy(notifications[i].schema, date, notifications[i].state, startDate)
if (result.length > 0) {
for (let p = 0; p < result.length; p++) {
await sendNotificationMails(result[p], daysUntilDeletionTemp)
}
}
}
}
}
}

async function sendNotificationMails(report: ITravel | IExpenseReport | IHealthCareCost, daysUntilDeletion: number) {
if (report) {
let owner
owner = await User.findOne({ _id: report.owner._id }).lean()
if (owner) {
let recipients = [owner]

var reportType: ReportType
if (reportIsTravel(report)) {
reportType = 'travel'
} else if (reportIsHealthCareCost(report)) {
reportType = 'healthCareCost'
} else {
reportType = 'expenseReport'
}

const language = recipients[0].settings.language

const interpolation: { owner: string; lng: Locale; days: number; reportName: string } = {
owner: report.owner.name.givenName,
lng: language,
days: daysUntilDeletion,
reportName: report.name
}

const button = {
text: '',
link: ''
}

button.link = `${process.env.VITE_FRONTEND_URL}/${reportType}/${report._id}`

button.text = i18n.t('labels.viewX', { lng: language, X: i18n.t(`labels.${reportType}`, { lng: language }) })

const subject = i18n.t(`mail.${reportType}.${report.state}DeletedSoon.subject`, interpolation)
const paragraph = i18n.t(`mail.${reportType}.${report.state}DeletedSoon.paragraph`, interpolation)
await sendMail(recipients, subject, paragraph, button, '')
}
}
}
async function getSettings() {
return await Settings.findOne({}).lean()
}

export async function retentionPolicy() {
const settings = await getSettings()
if (settings) {
await notificationMailForDeletions(settings.retentionPolicy)
await triggerDeletion(settings.retentionPolicy)
} else {
console.error('Settings not found!')
}
}
3 changes: 3 additions & 0 deletions backend/server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import { projectSchema } from './models/project.js'
import { settingsSchema } from './models/settings.js'
import { UserDoc, userSchema } from './models/user.js'
import { generateForms } from './models/vueformGenerator.js'
import { retentionPolicy } from './retentionpolicy.js'
const port = parseInt(process.env.BACKEND_PORT)
const url = process.env.VITE_BACKEND_URL

Expand Down Expand Up @@ -83,3 +84,5 @@ app.listen(port, () => {

// Update lump sums every day at 1 AM
CronJob.from({ cronTime: '0 1 * * *', onTick: fetchAndUpdateLumpSums, start: true })
// Trigger automatic deletion and notification mails for upcoming deletions every day at 1 AM
CronJob.from({ cronTime: '0 1 * * *', onTick: retentionPolicy, start: true })
Loading

0 comments on commit 072ed8c

Please sign in to comment.