{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":727772073,"defaultBranch":"main","name":"tabular-cdc-bootstrapper","ownerLogin":"tabular-io","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2023-12-05T14:44:12.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/81718086?v=4","public":true,"private":false,"isOrgOwned":true},"refInfo":{"name":"","listCacheKey":"v0:1707216802.0","currentOid":""},"activityList":{"items":[{"before":"8f624b83796c4016b36dc7b630ff60c128ab90c9","after":"5e168b5a0bbfae39a088e4fef901a1c5c4475de7","ref":"refs/heads/main","pushedAt":"2024-02-07T00:31:39.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"randypitcherii","name":"Randy","path":"/randypitcherii","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8570256?s=80&v=4"},"commit":{"message":"Feature/slimdown (#19)\n\n* deleted that pesky serverless logic\r\n\r\n* added better error handling in bootstrapper to skip errored targets\r\n\r\n* simplified the tabular module by assuming target paths are well generated with basic validity checks\r\n\r\n* removed unneeded dependencies\r\n\r\n* right sized dependencies\r\n\r\n* implemented cdc target table creation\r\n\r\n* implemented full cdc bootstrapping! Tested manually, with pytests to follow\r\n\r\n* added sample dms data generation\r\n\r\n* updated docs\r\n\r\n* implemented good sample data gen, updated test functions\r\n\r\n* removed cdc dependent-tables quoting that broke cdc\r\n\r\n* changed sample data column from 'timestamp' to 'ts' to avoid reserved words in compute engines\r\n\r\n* storing current status\r\n\r\n* implemented a much slimmer bootstrapper. You give s3 paths, it boostraps. NO CDC YET THOUGH\r\n\r\n* removed the PR test since I deleted the pytest work\r\n\r\n* implemented CDC config","shortMessageHtmlLink":"Feature/slimdown (#19)"}},{"before":"bc65ee6fab365138362ac2e03d22a273e3826fd8","after":null,"ref":"refs/heads/feature/slimdown","pushedAt":"2024-02-07T00:31:39.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"randypitcherii","name":"Randy","path":"/randypitcherii","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8570256?s=80&v=4"}},{"before":"d163b7b2db051d4c10ff7c2875a0dc1ba7b461d0","after":"bc65ee6fab365138362ac2e03d22a273e3826fd8","ref":"refs/heads/feature/slimdown","pushedAt":"2024-02-07T00:31:31.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"randypitcherii","name":"Randy","path":"/randypitcherii","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8570256?s=80&v=4"},"commit":{"message":"resolved conflict","shortMessageHtmlLink":"resolved conflict"}},{"before":"1f842dd907ed2ecd7280d70828a45f8c4f7cddeb","after":"d163b7b2db051d4c10ff7c2875a0dc1ba7b461d0","ref":"refs/heads/feature/slimdown","pushedAt":"2024-02-07T00:29:13.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"randypitcherii","name":"Randy","path":"/randypitcherii","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8570256?s=80&v=4"},"commit":{"message":"implemented CDC config","shortMessageHtmlLink":"implemented CDC config"}},{"before":"15d9d4fb5e700f650cfaeb942f89ba8353871067","after":"1f842dd907ed2ecd7280d70828a45f8c4f7cddeb","ref":"refs/heads/feature/slimdown","pushedAt":"2024-02-06T23:49:26.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"randypitcherii","name":"Randy","path":"/randypitcherii","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8570256?s=80&v=4"},"commit":{"message":"removed the PR test since I deleted the pytest work","shortMessageHtmlLink":"removed the PR test since I deleted the pytest work"}},{"before":"71faccc7b44ac44d28769941c9fa9063afefaaa9","after":"15d9d4fb5e700f650cfaeb942f89ba8353871067","ref":"refs/heads/feature/slimdown","pushedAt":"2024-02-06T23:48:34.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"randypitcherii","name":"Randy","path":"/randypitcherii","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8570256?s=80&v=4"},"commit":{"message":"implemented a much slimmer bootstrapper. You give s3 paths, it boostraps. NO CDC YET THOUGH","shortMessageHtmlLink":"implemented a much slimmer bootstrapper. You give s3 paths, it boostr…"}},{"before":null,"after":"71faccc7b44ac44d28769941c9fa9063afefaaa9","ref":"refs/heads/feature/slimdown","pushedAt":"2024-02-06T10:53:22.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"randypitcherii","name":"Randy","path":"/randypitcherii","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8570256?s=80&v=4"},"commit":{"message":"storing current status","shortMessageHtmlLink":"storing current status"}},{"before":"b9552cc6e4ead3e84d4f3171f88080e02b1a392c","after":"71faccc7b44ac44d28769941c9fa9063afefaaa9","ref":"refs/heads/feature/16--bootstrap-cdc-mirror-target","pushedAt":"2024-02-06T10:53:05.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"randypitcherii","name":"Randy","path":"/randypitcherii","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8570256?s=80&v=4"},"commit":{"message":"storing current status","shortMessageHtmlLink":"storing current status"}},{"before":"3da1835bc81ef44369bf10039bf906ca7bed2351","after":"b9552cc6e4ead3e84d4f3171f88080e02b1a392c","ref":"refs/heads/feature/16--bootstrap-cdc-mirror-target","pushedAt":"2023-12-13T18:30:11.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"randypitcherii","name":"Randy","path":"/randypitcherii","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8570256?s=80&v=4"},"commit":{"message":"changed sample data column from 'timestamp' to 'ts' to avoid reserved words in compute engines","shortMessageHtmlLink":"changed sample data column from 'timestamp' to 'ts' to avoid reserved…"}},{"before":null,"after":"3da1835bc81ef44369bf10039bf906ca7bed2351","ref":"refs/heads/feature/16--bootstrap-cdc-mirror-target","pushedAt":"2023-12-13T18:28:36.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"randypitcherii","name":"Randy","path":"/randypitcherii","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8570256?s=80&v=4"},"commit":{"message":"removed cdc dependent-tables quoting that broke cdc","shortMessageHtmlLink":"removed cdc dependent-tables quoting that broke cdc"}},{"before":"7637fbe4a47dc2450752be595839c5362a73b160","after":null,"ref":"refs/heads/feature/16--bootstrap-cdc-mirror-target","pushedAt":"2023-12-13T15:20:30.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"randypitcherii","name":"Randy","path":"/randypitcherii","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8570256?s=80&v=4"}},{"before":"19cccaa2ef53208802cc11a4afdde34be5cb125b","after":"8f624b83796c4016b36dc7b630ff60c128ab90c9","ref":"refs/heads/main","pushedAt":"2023-12-13T15:20:29.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"randypitcherii","name":"Randy","path":"/randypitcherii","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8570256?s=80&v=4"},"commit":{"message":"Bootstrap CDC Target Table (#18)\n\n* deleted that pesky serverless logic\r\n\r\n* added better error handling in bootstrapper to skip errored targets\r\n\r\n* simplified the tabular module by assuming target paths are well generated with basic validity checks\r\n\r\n* removed unneeded dependencies\r\n\r\n* right sized dependencies\r\n\r\n* implemented cdc target table creation\r\n\r\n* implemented full cdc bootstrapping! Tested manually, with pytests to follow\r\n\r\n* added sample dms data generation\r\n\r\n* updated docs\r\n\r\n* implemented good sample data gen, updated test functions","shortMessageHtmlLink":"Bootstrap CDC Target Table (#18)"}},{"before":"1daec360ef9e2355ace675259a74b4ba15554882","after":"7637fbe4a47dc2450752be595839c5362a73b160","ref":"refs/heads/feature/16--bootstrap-cdc-mirror-target","pushedAt":"2023-12-12T20:08:53.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"randypitcherii","name":"Randy","path":"/randypitcherii","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8570256?s=80&v=4"},"commit":{"message":"implemented good sample data gen, updated test functions","shortMessageHtmlLink":"implemented good sample data gen, updated test functions"}},{"before":"25ecc4039afaf2217b5e0fb0f07125e86aabebf1","after":"1daec360ef9e2355ace675259a74b4ba15554882","ref":"refs/heads/feature/16--bootstrap-cdc-mirror-target","pushedAt":"2023-12-12T15:32:19.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"randypitcherii","name":"Randy","path":"/randypitcherii","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8570256?s=80&v=4"},"commit":{"message":"updated docs","shortMessageHtmlLink":"updated docs"}},{"before":"5184adbb2315faad72e7620b366bd0bf3dfda7a0","after":"25ecc4039afaf2217b5e0fb0f07125e86aabebf1","ref":"refs/heads/feature/16--bootstrap-cdc-mirror-target","pushedAt":"2023-12-12T14:55:51.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"randypitcherii","name":"Randy","path":"/randypitcherii","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8570256?s=80&v=4"},"commit":{"message":"added sample dms data generation","shortMessageHtmlLink":"added sample dms data generation"}},{"before":"fa347b17aa2006aabf3ee19ee7dc50ed81d6d5be","after":"5184adbb2315faad72e7620b366bd0bf3dfda7a0","ref":"refs/heads/feature/16--bootstrap-cdc-mirror-target","pushedAt":"2023-12-11T21:30:09.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"randypitcherii","name":"Randy","path":"/randypitcherii","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8570256?s=80&v=4"},"commit":{"message":"implemented full cdc bootstrapping! Tested manually, with pytests to follow","shortMessageHtmlLink":"implemented full cdc bootstrapping! Tested manually, with pytests to …"}},{"before":"00e0c83ac1f0cfb86d01fd2fffbcf57d61e702f0","after":"fa347b17aa2006aabf3ee19ee7dc50ed81d6d5be","ref":"refs/heads/feature/16--bootstrap-cdc-mirror-target","pushedAt":"2023-12-11T18:44:25.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"randypitcherii","name":"Randy","path":"/randypitcherii","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8570256?s=80&v=4"},"commit":{"message":"implemented cdc target table creation","shortMessageHtmlLink":"implemented cdc target table creation"}},{"before":"635bfc67ff79f7ece3584e6719a6a45b7c7e333d","after":"00e0c83ac1f0cfb86d01fd2fffbcf57d61e702f0","ref":"refs/heads/feature/16--bootstrap-cdc-mirror-target","pushedAt":"2023-12-11T16:17:57.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"randypitcherii","name":"Randy","path":"/randypitcherii","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8570256?s=80&v=4"},"commit":{"message":"right sized dependencies","shortMessageHtmlLink":"right sized dependencies"}},{"before":"16fc8c1b55f8ca2d69f729d2b790f1da02fbf006","after":"635bfc67ff79f7ece3584e6719a6a45b7c7e333d","ref":"refs/heads/feature/16--bootstrap-cdc-mirror-target","pushedAt":"2023-12-11T16:11:50.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"randypitcherii","name":"Randy","path":"/randypitcherii","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8570256?s=80&v=4"},"commit":{"message":"removed unneeded dependencies","shortMessageHtmlLink":"removed unneeded dependencies"}},{"before":"464647f9382236268cd434a1a5974fb28ee285c9","after":"16fc8c1b55f8ca2d69f729d2b790f1da02fbf006","ref":"refs/heads/feature/16--bootstrap-cdc-mirror-target","pushedAt":"2023-12-11T16:03:59.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"randypitcherii","name":"Randy","path":"/randypitcherii","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8570256?s=80&v=4"},"commit":{"message":"simplified the tabular module by assuming target paths are well generated with basic validity checks","shortMessageHtmlLink":"simplified the tabular module by assuming target paths are well gener…"}},{"before":"f5d299a1e2f1d9e73820bd85cfbfbac47eeadf42","after":"464647f9382236268cd434a1a5974fb28ee285c9","ref":"refs/heads/feature/16--bootstrap-cdc-mirror-target","pushedAt":"2023-12-11T15:26:26.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"randypitcherii","name":"Randy","path":"/randypitcherii","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8570256?s=80&v=4"},"commit":{"message":"added better error handling in bootstrapper to skip errored targets","shortMessageHtmlLink":"added better error handling in bootstrapper to skip errored targets"}},{"before":"19cccaa2ef53208802cc11a4afdde34be5cb125b","after":"f5d299a1e2f1d9e73820bd85cfbfbac47eeadf42","ref":"refs/heads/feature/16--bootstrap-cdc-mirror-target","pushedAt":"2023-12-11T15:17:39.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"randypitcherii","name":"Randy","path":"/randypitcherii","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8570256?s=80&v=4"},"commit":{"message":"deleted that pesky serverless logic","shortMessageHtmlLink":"deleted that pesky serverless logic"}},{"before":null,"after":"19cccaa2ef53208802cc11a4afdde34be5cb125b","ref":"refs/heads/feature/16--bootstrap-cdc-mirror-target","pushedAt":"2023-12-11T15:12:00.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"randypitcherii","name":"Randy","path":"/randypitcherii","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8570256?s=80&v=4"},"commit":{"message":"Dockerize + Polling model (#15)\n\n* fixed a painful bloated dependencies issue\r\n\r\n* wired up table creation -- next is simple table property management\r\n\r\n* STUPID - I didn't have .github in the project root\r\n\r\n* specified python version better for gh action\r\n\r\n* made pipenv just a little more flexible\r\n\r\n* made gh actions less strict\r\n\r\n* remember to pipenv install dev deps\r\n\r\n* added envs to gh action workflow\r\n\r\n* fixed bad var ref for ci\r\n\r\n* fixed state issues in unit testing\r\n\r\n* refactor to s3_monitoring_uri syntax, split table creation into loader tables and cdc mirror tables\r\n\r\n* small tweak to config file\r\n\r\n* added s3 uri indicator\r\n\r\n* largest commit ever, but batch is working with duckdb 🦆\r\n\r\n* little structure switcharoo\r\n\r\n* adding docker building\r\n\r\n* finished dockerization\r\n\r\n* got docker working\r\n\r\n* implemented testing and proper duck sql for batch processor\r\n\r\n* updated actions workflow\r\n\r\n* added tweaked ci test command to batch runner\r\n\r\n* forgot to update the action 🫠\r\n\r\n* odd input error arg to docker test command\r\n\r\n* added env var passing for docker testing\r\n\r\n* pipfile refresh\r\n\r\n* debug step for test-ci for batch\r\n\r\n* more debugging\r\n\r\n* deleted the lambda approach\r\n\r\n* moved test_tabular back\r\n\r\n* fixed bugs and fixed tests! Man, I was on something else last night when I write this 😬\r\n\r\n* ironed out new bugs in batch_bootstrapper\r\n\r\n* got more surgical with docker env mirroring for ci job\r\n\r\n* added env var printing, except the secret, to the gha\r\n\r\n* fixed bad .env filename\r\n\r\n* fixed outdated env vars in gha\r\n\r\n* hopefully final touch on .env file creation\r\n\r\n* added important cleanup step to table creation tests\r\n\r\n* parallelized pytest, simplified makefile to a single make test command\r\n\r\n* one missing resolution","shortMessageHtmlLink":"Dockerize + Polling model (#15)"}},{"before":"beb742fc367ef470ef90e629f9c91920307c6cfc","after":null,"ref":"refs/heads/feature/10--dockerize","pushedAt":"2023-12-11T14:59:34.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"randypitcherii","name":"Randy","path":"/randypitcherii","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8570256?s=80&v=4"}},{"before":"8dce317084d76d83707e42d2d4e55e46ff91c5bb","after":"19cccaa2ef53208802cc11a4afdde34be5cb125b","ref":"refs/heads/main","pushedAt":"2023-12-11T14:59:33.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"randypitcherii","name":"Randy","path":"/randypitcherii","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8570256?s=80&v=4"},"commit":{"message":"Dockerize + Polling model (#15)\n\n* fixed a painful bloated dependencies issue\r\n\r\n* wired up table creation -- next is simple table property management\r\n\r\n* STUPID - I didn't have .github in the project root\r\n\r\n* specified python version better for gh action\r\n\r\n* made pipenv just a little more flexible\r\n\r\n* made gh actions less strict\r\n\r\n* remember to pipenv install dev deps\r\n\r\n* added envs to gh action workflow\r\n\r\n* fixed bad var ref for ci\r\n\r\n* fixed state issues in unit testing\r\n\r\n* refactor to s3_monitoring_uri syntax, split table creation into loader tables and cdc mirror tables\r\n\r\n* small tweak to config file\r\n\r\n* added s3 uri indicator\r\n\r\n* largest commit ever, but batch is working with duckdb 🦆\r\n\r\n* little structure switcharoo\r\n\r\n* adding docker building\r\n\r\n* finished dockerization\r\n\r\n* got docker working\r\n\r\n* implemented testing and proper duck sql for batch processor\r\n\r\n* updated actions workflow\r\n\r\n* added tweaked ci test command to batch runner\r\n\r\n* forgot to update the action 🫠\r\n\r\n* odd input error arg to docker test command\r\n\r\n* added env var passing for docker testing\r\n\r\n* pipfile refresh\r\n\r\n* debug step for test-ci for batch\r\n\r\n* more debugging\r\n\r\n* deleted the lambda approach\r\n\r\n* moved test_tabular back\r\n\r\n* fixed bugs and fixed tests! Man, I was on something else last night when I write this 😬\r\n\r\n* ironed out new bugs in batch_bootstrapper\r\n\r\n* got more surgical with docker env mirroring for ci job\r\n\r\n* added env var printing, except the secret, to the gha\r\n\r\n* fixed bad .env filename\r\n\r\n* fixed outdated env vars in gha\r\n\r\n* hopefully final touch on .env file creation\r\n\r\n* added important cleanup step to table creation tests\r\n\r\n* parallelized pytest, simplified makefile to a single make test command\r\n\r\n* one missing resolution","shortMessageHtmlLink":"Dockerize + Polling model (#15)"}},{"before":"be1ae5b2c7f9e5ce0cb2cf3f2964e5f630925dde","after":"beb742fc367ef470ef90e629f9c91920307c6cfc","ref":"refs/heads/feature/10--dockerize","pushedAt":"2023-12-11T14:56:30.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"randypitcherii","name":"Randy","path":"/randypitcherii","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8570256?s=80&v=4"},"commit":{"message":"one missing resolution","shortMessageHtmlLink":"one missing resolution"}},{"before":"dabc21b6b34da61d6ad2564dca5f92bd2c0d7e4f","after":"be1ae5b2c7f9e5ce0cb2cf3f2964e5f630925dde","ref":"refs/heads/feature/10--dockerize","pushedAt":"2023-12-11T14:54:36.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"randypitcherii","name":"Randy","path":"/randypitcherii","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8570256?s=80&v=4"},"commit":{"message":"resolved merge conflicts","shortMessageHtmlLink":"resolved merge conflicts"}},{"before":"6e7c5d0be207862bf1edbc194a2e561e6c09ad66","after":"dabc21b6b34da61d6ad2564dca5f92bd2c0d7e4f","ref":"refs/heads/feature/10--dockerize","pushedAt":"2023-12-11T14:52:12.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"randypitcherii","name":"Randy","path":"/randypitcherii","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8570256?s=80&v=4"},"commit":{"message":"parallelized pytest, simplified makefile to a single make test command","shortMessageHtmlLink":"parallelized pytest, simplified makefile to a single make test command"}},{"before":"25d92cea68918ffbc89ca3ef823dea2aac6e9b9d","after":"6e7c5d0be207862bf1edbc194a2e561e6c09ad66","ref":"refs/heads/feature/10--dockerize","pushedAt":"2023-12-11T14:50:09.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"randypitcherii","name":"Randy","path":"/randypitcherii","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8570256?s=80&v=4"},"commit":{"message":"added important cleanup step to table creation tests","shortMessageHtmlLink":"added important cleanup step to table creation tests"}},{"before":"4f6c8eb1c483b1479cdca72d91119f2e14179a39","after":null,"ref":"refs/heads/feature/05--configure-fileloader-and-cdc-props","pushedAt":"2023-12-11T14:40:17.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"randypitcherii","name":"Randy","path":"/randypitcherii","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8570256?s=80&v=4"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"startCursor":"Y3Vyc29yOnYyOpK7MjAyNC0wMi0wN1QwMDozMTozOS4wMDAwMDBazwAAAAPz6uLT","endCursor":"Y3Vyc29yOnYyOpK7MjAyMy0xMi0xMVQxNDo0MDoxNy4wMDAwMDBazwAAAAPHWIOU"}},"title":"Activity · tabular-io/tabular-cdc-bootstrapper"}