Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move to libzim9 #1838

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

Move to libzim9 #1838

wants to merge 7 commits into from

Conversation

kelvinhammond
Copy link
Collaborator

No description provided.

@kelson42
Copy link
Collaborator

@kelvinhammond Thank you again for being back on this super-important PR. Just wanted to ask what is the status? Do you think this is ready to review? Do we just need to release latest node-libzim to get the CI passing?

@kelvinhammond
Copy link
Collaborator Author

CI build fails but looks good to review to me @kelson42

@kelson42
Copy link
Collaborator

kelson42 commented May 21, 2023

@kelvinhammond I had a look and there a bit of work related to links and content. I will work on this on your PR and come back to you.

@kelson42 kelson42 requested review from kelson42 and removed request for rgaudin and pavel-karatsiuba May 21, 2023 06:44
Copy link
Collaborator

@kelson42 kelson42 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had a look to the PR. Here are my comments:

  • MWoffliner builds a ZIM file, which is a good start
  • The ZIM file built is incorrect by many way, you can see it easily if you open it with kiwix-serve
  • One problem is that new ZIM flies (using latest libzim) do handle the ZIM namespaces differently. They were before open and free to use and now they are hidden behind the scene in the libzim. We did that to basically how the writting of content for the ZIM developer in the C namespace. This means that everything should be in the C namespace now (transparent for MWoffliner) whereas in the past we have put the images in the I namespace and the article in the A namespace. All of this has a few implication how we writte the content and how we create the links in the HTML. I could handle this kind of problem.
  • The other problem is that it seems the libzim creates corrupted content. I have tried with the "Animation" article in the Wikipedia in English. It seems that many images are corrupted, for example FelixTheCat-1919-FelineFollies_silent.ogv.180p.vp9.webm which is now not a valid webm file. I suspect the reason is that you write anything through the StringItem which is IMO not appropriate for blobs.

IMO we should really fix the problem with blobs in priority.

@stale
Copy link

stale bot commented May 28, 2023

This pull request has been automatically marked as stale because it has not had recent activity. It will be now be reviewed manually. Thank you for your contributions.

@stale stale bot added the stale label May 28, 2023
Data is lost on buffer.toString
@stale stale bot removed the stale label Jun 7, 2023
@kelson42 kelson42 changed the title Move to libzim8 Move to libzim9 Mar 5, 2024
@kelson42
Copy link
Collaborator

kelson42 commented Mar 5, 2024

We should now rebase on latest HEAD now that main branch is ready for this move and now that node-libzim has been fixed

@kelson42
Copy link
Collaborator

kelson42 commented Jun 24, 2024

@audiodude Please have a look to this PR, this has been done before the rearchitecturing of Fall 2023, but otherwise most of the code should be OK and just to be taken.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants