Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[UI] Import Jupyter notebook #369

Merged
merged 6 commits into from
Jul 14, 2023
Merged

Conversation

senwang86
Copy link
Collaborator

@senwang86 senwang86 commented Jul 12, 2023

Summary

  • Add a right click menu item to import a single Jupyter notebook, both code cells and markdown cells are rendered correctly.
  • All cells are grouped in a single column within a single scope.
  • A notebook can be imported more than once.
  • The notebook can be imported from inside the scope.

Test

importJupyterNotebook

Follow-up

  • More tests by importing other public Github notebooks
  • Another round of the "export" test for the content integrity.

Copy link
Collaborator

@lihebi lihebi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Attaching a few minor comments.

// NOTE for import ipynb: we need to specify some reasonable height so that
// the imported pods can be properly laid-out. 130 is a good one.
// This number MUST match the number in Canvas.tsx (refer to "A BIG HACK" in Canvas.tsx).
height: 130,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This magic number 130 must be kept consistent with the 130 in Canvas.tsx. Let's store it in one global variable and access it in both files. Also, we can use some floating number to mark it special, e.g., 136.66.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given it's a hacky way to apply autolayout, I think we can use 130 here. We need to find a better way for autolayer eventually.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given it's a hacky way

Precisely because it is hacky, we want to avoid breaking it accidentally, by:

  • using a global variable to avoid potential inconsistency error
  • using some floating numbers to prevent a pod from being the same value.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see you have changed to a global variable. The floating number isn't that critical (as long as it's not 120).

dirty: true,
pending: true,
});
get().moveIntoScope(node.id, scopeNode.id);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current moveIntoScope function is pretty heavy because it calls the three heavy functions adjustLevel(), updateView, and buildNode2Children at its end (ref). I can see it to be very slow when a repo becomes large.

Those three functions only need to be called once at the end of the ipynbImport action. So let's add a moveIntoScopeLight without those three functions and use it here.

moveIntoScopeLight() {...}
moveIntoScope() {
    moveIntoScopeLight(); 
    get().adjustLevel();
    get().updateView();
    get().buildNode2Children();
}
importIpynb() {
    ...
    for() {...
        moveIntoScopeLight()
    }
    get().adjustLevel();
    get().updateView();
    get().buildNode2Children();
    ...
}

@@ -1003,7 +1068,7 @@ export const createCanvasSlice: StateCreator<MyState, [], [], CanvasSlice> = (
/**
* Use d3-force to auto layout the nodes.
*/
autoLayout: (scopeId) => {
autoLayout: async (scopeId) => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was for debugging. Not needed.

@forrestbao
Copy link
Collaborator

Thanks @senwang86. Maybe the import menu can be moved to the navigation bar?

@senwang86
Copy link
Collaborator Author

Thanks @senwang86. Maybe the import menu can be moved to the navigation bar?

I initially placed the "import button" in the navigation bar, but later realized that the user may want to control "where" the imported notebook is inserted. In the end, I capture the right-click position on canvas and utilize it as the insertion position.

@forrestbao
Copy link
Collaborator

Thanks @senwang86. Maybe the import menu can be moved to the navigation bar?

I initially placed the "import button" in the navigation bar, but later realized that the user may want to control "where" the imported notebook is inserted. In the end, I capture the right-click position on canvas and utilize it as the insertion position.

Good thoughts.

@lihebi
Copy link
Collaborator

lihebi commented Jul 14, 2023

Thanks, Sen!

@lihebi lihebi merged commit 3a243fc into codepod-io:main Jul 14, 2023
@forrestbao
Copy link
Collaborator

Hi @senwang86, can I make a minor additional feature request? Can you allow .py scripts to be imported into Codepod? Just the text content.

@senwang86
Copy link
Collaborator Author

Hi @senwang86, can I make a minor additional feature request? Can you allow .py scripts to be imported into Codepod? Just the text content.

Sure, I believe we can begin by importing a .py file into a single pod. I'm wondering if the "local runtime" would be a better fit for the multiple file import case though.

@forrestbao
Copy link
Collaborator

forrestbao commented Jul 14, 2023

Yes, importing one .py file into a single pod is good enough for now. The reason I want to import .py file is because I can then see, modify and use its content in the same interface with the rest of the code in a Codepod repo. Changes can be quickly reflected in the runtime without needing to reimport a module.

@senwang86
Copy link
Collaborator Author

Yes, importing one .py file into a single pod is good enough for now. The reason I want to import .py file is because I can then see, modify and use its content in the same interface with the rest of the code in a Codepod repo. Changes can be quickly reflected in the runtime without needing to reimport a module.

Please check #374 .

@forrestbao
Copy link
Collaborator

@senwang86, please add support to import execution output from a ipynb when you have time

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants