I was on this team at SingleStore and I can vouch for how hard this team worked on this project. I just opened a couple notebooks in production and they loaded *instantly*, so kudos to the team for seeing this project through.
(If you're not familiar with SingleStore's Jupyter Notebooks, they're similar to Databricks Notebooks[1] or Azure Synapse Notebooks[2]).
I dislike how Jupyter notebooks have become normalized. Yes, the interactive execution and visuals are nice for more academic workflows where the priority is quick results over code organization. However, when it comes to sharing code with others for the sake of doing reproducible science, jupyter notebooks cause more trouble than they are worth. Using cell based execution with python is so elegant with '# %%' lines in regular .py files (though it requires using VSCode or fiddling with vim plugins which not all scientists want to do I suppose). No .ipynb is necessary, .py files can be version controlled and shared like normal code while sill retaining the ability to use interactively, cell by cell.
Its much easier to organize .py files into a proper python module, and then share and collaborate with others. Instead, groups will collect jumbles of slightly different versions of the same jupyter notebooks that progressively become more complex and less manageable over time. It's not a hypothetical unfortunately, I've seen this happen at major university labs. I'm not blaming anyone because I understand -- the funding is there to do science and not rewrite code to build convenient software libraries. Yet, I can't help but wish jupyter notebooks could be removed from academic workflows.
While we're on the topic of jupyter enhancements, would really love to be able to pop a cell off the pending execution stack if I realized running it would be a mistake and still have time before it gets there.. :^)
davidgomes ·126 days ago
(If you're not familiar with SingleStore's Jupyter Notebooks, they're similar to Databricks Notebooks[1] or Azure Synapse Notebooks[2]).
[1]: https://docs.databricks.com/en/notebooks/index.html
[2]: https://learn.microsoft.com/en-us/azure/synapse-analytics/sp...
Show replies
spiralk ·126 days ago
Its much easier to organize .py files into a proper python module, and then share and collaborate with others. Instead, groups will collect jumbles of slightly different versions of the same jupyter notebooks that progressively become more complex and less manageable over time. It's not a hypothetical unfortunately, I've seen this happen at major university labs. I'm not blaming anyone because I understand -- the funding is there to do science and not rewrite code to build convenient software libraries. Yet, I can't help but wish jupyter notebooks could be removed from academic workflows.
Show replies
pizza ·126 days ago
Show replies
paddy_m ·126 days ago
What mix of JS packages do you see? Could that be built into one uber package?
Show replies
dkga ·126 days ago
Show replies