Plan of attack

Design

• json:
  • blogs
    • title, URLs, categories
  • => Cf issue #1
• how to cache downloaded feeds? Memory? Filesystem?
  • Decision: local xml files, separate fetcher for now (maybe just use wget?)
• How to leverage http cache settings
  • only fetch header, compare against some measure, download feed
  • if HTTP header has cache-control => max-age or last-modified, use that
  • if not (BR :-( ), then what?
  • won't work. Ignore for now.
• generate category pages
  • filter json by category
  • apply pagewriter.js
• generate general post page
  • don't filter
  • apply pagewriter.js
  • performance hit: we're doing it twice
• front page: category page for twitter feed of editor picks
  • use editor.js to search twitter and create feed and frontpage simultaneously
• maybe iterate towards:
  • create feed object for each category and all-posts page
  • go through feeds and stream posts into "open" feedobjects
  • at the end of all feedprocesing, finish the feedobjects pass them to pagewriter.js
  • and pass the global feed to pagewriter
• import data from mathblogging into json
• we need dates in the json
  • https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/parse
  • Date.parse('2014-03-12T13:37:27+00:00') > Date.parse('2014-02-12T13:37:27+00:00')
  • how to read http headers
    • http://stackoverflow.com/questions/5922842/getting-http-headers-with-node-js
    • https://github.com/request/request
• revisit es6
• truncate actual article content: https://github.com/huang47/nodejs-html-truncate
• testing: https://medium.com/javascript-scene/why-i-use-tape-instead-of-mocha-so-should-you-6aa105d8eaf4

---

• We haven't tackled the "all posts" yet, both feed and page. Perhaps we should redesign things to build a global feed first and use that to build all pages and feeds? I.e., combine all feeds, tag each item with its categories etc.